typeerror: a bytes-like object is required, not 'str'

typeerror: a bytes-like object is required, not 'str'

This is a very common type of error faced by programmers while coding in Python. The typeerror occurs when there is a mismatch of data types. It happens when you run the same code on different versions of Python. For example, if you are coding in Python 2, you will not face problems while handling bytes and Unicode characters. But problems may arise when you are running the same code in Python 3.

Example 1

# Import gzip module
import gzip

# initialize line_count variable 
total_lines=0

# Open file with gzip module
with gzip.open('myfile.gz', 'rb') as fin:
        for line in fin:
            if (total_lines == 0):
                total_lines+=1
            else:
                number_columns = line.strip('\n').split('\t')
                print('Print File Rows: ',number_columns);

Output:

    number_columns = line.strip('\n').split('\t')
TypeError: a bytes-like object is required, not 'str'

This code works without any errors in Python 2 and counts the total number of lines in the file. But the same code when executed in Python 3 will throw the error - typeerror: a bytes-like object is required, not 'str'. This is because, in Python 2, the strings are by default treated as bytes. The original strings in Python 2 are 8-bit strings, which play a crucial role while working with byte sequences and ASCII text. This version also supports automatic coercion between bytes and Unicode objects.

But in Python 3, by default, the strings are treated as Unicode. But unlike Python 2 there is no facility of automatic type coercion between Unicode strings and bytes. So in the code mentioned above, when you are trying to open the file in binary mode, Python 3 throws an error.

How to fix this error?

The following code will fix this error:

# Import gzip module
import gzip

# initialize line_count variable 
total_lines=0

# Open file with gzip module
with gzip.open('myfile.sql.gz', 'rb') as fin:
        for line in fin:
            if (total_lines == 0):
                total_lines+=1
            else:
                number_columns = line.decode('utf8').strip('\n').split('\t')
                print('Print File Rows: ',number_columns);

In the code mentioned above, the decode() method decodes the lines in the text that help Python 3 to read them.

number_columns = line.decode('utf8').strip('\n').split('\t')

Let us look at another example where the typeerror might occur.

# Read file line by line
with open('myfile.txt', 'rb') as f:
    file_lines = [x.strip() for x in f.readlines()]

for file_line in file_lines:
    line_data = file_line.strip().lower()
    if 'some-pattern' in line_data: continue
print('Print file data: ',line_data)

In the above code, the “TypeError: a bytes-like object is required, not 'str'” is thrown at the last line when executed in Python 3. In the line, “with open('myfile.txt', 'rb') as f”, the file is opened in binary mode. So the entire text is returned in the form of bytes and not strings.

So, in the line “if 'some-pattern' in line_data: continue” the pattern test fails. You need to use a byte object to test the line_data variable.

The solution is:

# Read file line by line
with open('myfile.txt', 'rb') as f:
    file_lines = [x.strip() for x in f.readlines()]

for file_line in file_lines:
line_data = file_line.strip().lower()
if b'some-pattern' in line_data: continue
print('Print file data: ',line_data)

The last line now has a b flag for specifying a byte object. Additionally, you can also open the file in “r” or read mode instead of rb mode.

 if b'some-pattern' in line_data: continue

Conclusion 

It depends upon which version of Python you are currently using. If you are working in older versions such as Python 2.7 or 2.3, this error might not occur. But if you are thinking of updating your Python version and transferring your code to Python 3, keep the pointers mentioned above in mind.