Cannot Load Pickled Object

Cannot Load Pickled Object

The problem I am having is when I try to load the pickled object. I have tried using both pickle.loads and pickle.load Here are the results:

pickle.loads:

TypeError: 'str' does not support the buffer interface

pickle.load:

TypeError: file must have 'read' and 'readline' attributes

Can someone please tell me what I am doing wrong in this process?

elif str(parser) == "SwissWithdrawn_Parser":
    # swissprot name changes
    print("Gathering SwissProt update info...")
    cache_hits = 0
    cache_misses = 0
    files = set()

    for f in os.listdir("out/cache/"):
        if os.path.isfile("out/cache/" + f):
            files.add(f)

    for name in sp_lost_names:

        cached = False
        url = (
            ""
            + name
            + "+active%3ayes&format=tab&columns=entry%20name"
        )
        hashed_url = str(hash(url))

        ################### For Testing Only - use cache ##################
        if hashed_url in files:
            cached = True
            cache_hits += 1
            content = pickle.loads("out/cache/" + hashed_url)  # <-- problematic line
        else:
            cache_misses += 1
            content = urllib.request.urlopen(url)

        # get the contents returned from the HTTPResponse object
        content_list = [x.decode().strip() for x in content.readlines()]
        if not cached:
            with open("out/cache/" + hashed_url, "wb") as fp:
                pickle.dump(content_list, fp)
        ####################################################################

        # no replacement
        if len(content_list) is 0:
            change_log["swiss-names"] = {name: "withdrawn"}
        # get the new name
        else:
            new_name = content_list[1]
            change_log["swiss-names"] = {name: new_name}

2 Answers

You need to either read the file first (as binary bytes) and use pickle.loads(), or pass an open file object to the pickle.load() command. The latter is preferable:

with open('out/cache/' +hashed_url, 'rb') as pickle_file:
    content = pickle.load(pickle_file)

Neither method supports loading a pickle from a filename.

0

If you happen to be porting python2 to 3 and run into this error, python2 and 3 handle bytes different leading to the requirement to open your file handle with the 'b' option. For instance in python2 open(file, 'r') as f: my_list = pickle.load(f) works , but not in python3. Instead you must open with open(file, 'rb') as f: my_list = pickle.load(f)

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Elena Rostova
Author

Elena Rostova

Elena Rostova holds a Master's degree in Public Health Journalism. She covers groundbreaking medical research, holistic wellness trends, mental health awareness, and nutritional science.