I am getting an error that I cannot seem to reproduce on a large dataset during multiprocessing. List comprehension like:
[shuffler1.shuffle().decode('utf-8') for i in range(repeats)]
creates a UnicodeDecodeError at a random point using a 30-60 letter string with normal letters. I cannot reproduce it because the following code seems to work:
temp = [shuffler1.shuffle() for i in range(repeats)]
try:
d1["shuffle"] = [x.decode('utf-8') for x in temp]
except:
print(temp)
raise ValueError()
Can you think of a reason this happens?
Cheers!
I am getting an error that I cannot seem to reproduce on a large dataset during multiprocessing. List comprehension like:
[shuffler1.shuffle().decode('utf-8') for i in range(repeats)]
creates a UnicodeDecodeError at a random point using a 30-60 letter string with normal letters. I cannot reproduce it because the following code seems to work:
temp = [shuffler1.shuffle() for i in range(repeats)]
try:
d1["shuffle"] = [x.decode('utf-8') for x in temp]
except:
print(temp)
raise ValueError()
Can you think of a reason this happens?
Cheers!