Working with Binary Data in Python 3

Learn about the built-in bytes type and how to create raw bytes to work with.

In this video, learn how to format in various formats including:

• Binary (base two)

• Octal (base eight)

• Decimal (base ten)

• Hexadecimal (base sixteen)

You will also learn how to add leading zeros and padding for alignment.

In addition to the `bytes` type, there is a `bytearray` type that is mutable and can be modified. Let's look at how to create a `bytearray` and what we can do with it.

Now that we are familiar with immutable `bytes` and mutable `bytearray` objects, let's look at reading and writing bytes to a buffer with the `io.BytesIO` class.

In this section we will learn about bitwise operators like AND, OR, XOR, ones complement, and shifting bits left and right. This video is a quick overview of bitwise operations.

Learn how the bitwise AND operation works with examples.

Learn how the bitwise OR operation works with examples.

Learn how the bitwise XOR operation works with examples.

Learn how the ones complement works.

Bit shifting is the last bitwise operation to cover. Learn how to shift bits left and right in this video.

Overview of the "Conversion" section of the course dedicated to ways of converting between data types.

Learn how to convert an integer like 16 or 987 or -2,000,000,000 to a bytes object.

Learn how to take a list of integers ranging from 0-255 and turn it into a bytes object.

Learn how to take a binary string of 1's and 0's like "11110000" and convert it into a bytes object.

Learn how to convert raw bytes into an integer object

Learn how to convert bytes to and from hexadecimal strings.

Learn how to write bytes to a file on disk.

Learn how to get information about a file like:

• Whether a file exists or not

• How big a file is

• Who is the owner

• What permissions does the file have

Learn how to read bytes from an existing file into a bytes object in memory.

Learn how to navigate around a file and go to specific positions to get data that you are after.

Learn what struct packing is and why you might want to use it.

Let's walk through an example of packing several variables into a binary struct.

Learn how to unpack raw bytes into a series of variables that you can use to work with.

Get an overview of what encoding is and some common encodings like:

• ASCII

• UTF-8

• Base64

Learn how to encode a string to raw bytes and how to decode bytes to a string object in Python.

Learn how to encode a string as bytes or decode bytes as a string with UTF-8 encoding.

Learn to encode and decode base64 with two different methods. One using the codecs package and one using the base64 package. Both are built-in to Python 3.

Attached here are several Python example scripts that utilize the things we learned throughout this course.

• diff.py - Do two files match?

• is_jpeg.py - Does the file have a JPEG binary signature?

• read_boot_sector.py - Inspect the first 512 bytes of a file

• find_ascii_in_binary.py - Identify ASCII characters in binary files

• create_stego_zip_jpg.py - Hide a zip file inside a JPEG

• extract_pngs.py - Extract PNGs from a file and put them in a pngs/ directory

Let's review all the primary topics we covered so you can be sure you did not miss anything important. This is also a chance to ensure you have a good understanding of the important pieces and see if there is anything you want to review before you complete the course.