How to fix encoding issue in a python script for spanish alphabet
0
This short script is to convert a CSV into a JSON. The CSV contains letters in spanish alphabet, which is still UTF-8 I believe. The script seems to have issue reading letters like ñ or é from the CSV.
After executing
python2 csvToJSON.py
This console returns 'UnicodeDecodeError: 'utf8' codec can't decode byte 0xed in position 11: invalid continuation byte'
# -*- coding: utf-8 -*-
import codecs
import csv
import json
csvfile = codecs.open('practice.csv', encoding='utf-8').read()
# csvfile = open('practice.csv', 'r')
jsonfile = open('file.json', 'w')
fieldnames = ("contraseña", "id")
reader = csv.DictReader( csvfile, fieldnames)
for row in reader:
json.dump(row, jsonfile)
jsonfile.write('n')
python unicode encoding
asked Dec 31 '18 at 3:27
OmarOmar
73
|
show 2 more comments
0
This short script is to convert a CSV into a JSON. The CSV contains letters in spanish alphabet, which is still UTF-8 I believe. The script seems to have issue reading letters like ñ or é from the CSV.
After executing
python2 csvToJSON.py
This console returns 'UnicodeDecodeError: 'utf8' codec can't decode byte 0xed in position 11: invalid continuation byte'
# -*- coding: utf-8 -*-
import codecs
import csv
import json
csvfile = codecs.open('practice.csv', encoding='utf-8').read()
# csvfile = open('practice.csv', 'r')
jsonfile = open('file.json', 'w')
fieldnames = ("contraseña", "id")
reader = csv.DictReader( csvfile, fieldnames)
for row in reader:
json.dump(row, jsonfile)
jsonfile.write('n')
python unicode encoding
asked Dec 31 '18 at 3:27
OmarOmar
73
It seems to be in one of the ISO-8859-x encodings not in UTF-8.
– Klaus D.
Dec 31 '18 at 3:39
The Python 2csvmodule used to have problems with UTF-8, not sure if they were ever fixed. Is there a good reason you're not using Python 3 which is the recommended and supported version of the language?
– tripleee
Dec 31 '18 at 4:32
I see various problems with your code. For one, thecsvlibrary expects an open file (file handle), not a string. Also, the value ofcsvfileis of typeunicode, which thecsvlibrary can't handle in Python 2; it operates onstronly. Furthermore, the string"contraseña"should beu"contraseña". And it's unclear which line causes the decode error; please show the full traceback.
– lenz
Dec 31 '18 at 9:47
And, as tripleee said, if you can, switch to Python 3. It will make your life much easier with respect to encodings.
– lenz
Dec 31 '18 at 9:48
@lenz The full traceback at the terminal is:Traceback (most recent call last): File "csvToJSON.py", line 12, in <module> for row in reader: File "/usr/lib/python3.6/csv.py", line 112, in __next__ row = next(self.reader) File "/usr/lib/python3.6/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 11: invalid continuation byteThe same error is produced with Python3
– Omar
Dec 31 '18 at 21:20
|
show 2 more comments
0
0
0
0
This short script is to convert a CSV into a JSON. The CSV contains letters in spanish alphabet, which is still UTF-8 I believe. The script seems to have issue reading letters like ñ or é from the CSV.
After executing
python2 csvToJSON.py
This console returns 'UnicodeDecodeError: 'utf8' codec can't decode byte 0xed in position 11: invalid continuation byte'
# -*- coding: utf-8 -*-
import codecs
import csv
import json
csvfile = codecs.open('practice.csv', encoding='utf-8').read()
# csvfile = open('practice.csv', 'r')
jsonfile = open('file.json', 'w')
fieldnames = ("contraseña", "id")
reader = csv.DictReader( csvfile, fieldnames)
for row in reader:
json.dump(row, jsonfile)
jsonfile.write('n')
python unicode encoding
asked Dec 31 '18 at 3:27
OmarOmar
73
This short script is to convert a CSV into a JSON. The CSV contains letters in spanish alphabet, which is still UTF-8 I believe. The script seems to have issue reading letters like ñ or é from the CSV.
After executing
python2 csvToJSON.py
This console returns 'UnicodeDecodeError: 'utf8' codec can't decode byte 0xed in position 11: invalid continuation byte'
# -*- coding: utf-8 -*-
import codecs
import csv
import json
csvfile = codecs.open('practice.csv', encoding='utf-8').read()
# csvfile = open('practice.csv', 'r')
jsonfile = open('file.json', 'w')
fieldnames = ("contraseña", "id")
reader = csv.DictReader( csvfile, fieldnames)
for row in reader:
json.dump(row, jsonfile)
jsonfile.write('n')
python unicode encoding
python unicode encoding
asked Dec 31 '18 at 3:27
OmarOmar
73
asked Dec 31 '18 at 3:27
OmarOmar
73
asked Dec 31 '18 at 3:27
OmarOmar
73
asked Dec 31 '18 at 3:27
OmarOmar
73
asked Dec 31 '18 at 3:27
OmarOmar
73
73
It seems to be in one of the ISO-8859-x encodings not in UTF-8.
– Klaus D.
Dec 31 '18 at 3:39
The Python 2csvmodule used to have problems with UTF-8, not sure if they were ever fixed. Is there a good reason you're not using Python 3 which is the recommended and supported version of the language?
– tripleee
Dec 31 '18 at 4:32
I see various problems with your code. For one, thecsvlibrary expects an open file (file handle), not a string. Also, the value ofcsvfileis of typeunicode, which thecsvlibrary can't handle in Python 2; it operates onstronly. Furthermore, the string"contraseña"should beu"contraseña". And it's unclear which line causes the decode error; please show the full traceback.
– lenz
Dec 31 '18 at 9:47
And, as tripleee said, if you can, switch to Python 3. It will make your life much easier with respect to encodings.
– lenz
Dec 31 '18 at 9:48
@lenz The full traceback at the terminal is:Traceback (most recent call last): File "csvToJSON.py", line 12, in <module> for row in reader: File "/usr/lib/python3.6/csv.py", line 112, in __next__ row = next(self.reader) File "/usr/lib/python3.6/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 11: invalid continuation byteThe same error is produced with Python3
– Omar
Dec 31 '18 at 21:20
|
show 2 more comments
It seems to be in one of the ISO-8859-x encodings not in UTF-8.
– Klaus D.
Dec 31 '18 at 3:39
The Python 2csvmodule used to have problems with UTF-8, not sure if they were ever fixed. Is there a good reason you're not using Python 3 which is the recommended and supported version of the language?
– tripleee
Dec 31 '18 at 4:32
I see various problems with your code. For one, thecsvlibrary expects an open file (file handle), not a string. Also, the value ofcsvfileis of typeunicode, which thecsvlibrary can't handle in Python 2; it operates onstronly. Furthermore, the string"contraseña"should beu"contraseña". And it's unclear which line causes the decode error; please show the full traceback.
– lenz
Dec 31 '18 at 9:47
And, as tripleee said, if you can, switch to Python 3. It will make your life much easier with respect to encodings.
– lenz
Dec 31 '18 at 9:48
@lenz The full traceback at the terminal is:Traceback (most recent call last): File "csvToJSON.py", line 12, in <module> for row in reader: File "/usr/lib/python3.6/csv.py", line 112, in __next__ row = next(self.reader) File "/usr/lib/python3.6/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 11: invalid continuation byteThe same error is produced with Python3
– Omar
Dec 31 '18 at 21:20
It seems to be in one of the ISO-8859-x encodings not in UTF-8.
– Klaus D.
Dec 31 '18 at 3:39
It seems to be in one of the ISO-8859-x encodings not in UTF-8.
– Klaus D.
Dec 31 '18 at 3:39
The Python 2
csv module used to have problems with UTF-8, not sure if they were ever fixed. Is there a good reason you're not using Python 3 which is the recommended and supported version of the language?– tripleee
Dec 31 '18 at 4:32
The Python 2
csv module used to have problems with UTF-8, not sure if they were ever fixed. Is there a good reason you're not using Python 3 which is the recommended and supported version of the language?– tripleee
Dec 31 '18 at 4:32
I see various problems with your code. For one, the
csv library expects an open file (file handle), not a string. Also, the value of csvfile is of type unicode, which the csv library can't handle in Python 2; it operates on str only. Furthermore, the string "contraseña" should be u"contraseña". And it's unclear which line causes the decode error; please show the full traceback.– lenz
Dec 31 '18 at 9:47
I see various problems with your code. For one, the
csv library expects an open file (file handle), not a string. Also, the value of csvfile is of type unicode, which the csv library can't handle in Python 2; it operates on str only. Furthermore, the string "contraseña" should be u"contraseña". And it's unclear which line causes the decode error; please show the full traceback.– lenz
Dec 31 '18 at 9:47
And, as tripleee said, if you can, switch to Python 3. It will make your life much easier with respect to encodings.
– lenz
Dec 31 '18 at 9:48
And, as tripleee said, if you can, switch to Python 3. It will make your life much easier with respect to encodings.
– lenz
Dec 31 '18 at 9:48
@lenz The full traceback at the terminal is:
Traceback (most recent call last): File "csvToJSON.py", line 12, in <module> for row in reader: File "/usr/lib/python3.6/csv.py", line 112, in __next__ row = next(self.reader) File "/usr/lib/python3.6/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 11: invalid continuation byte The same error is produced with Python3– Omar
Dec 31 '18 at 21:20
@lenz The full traceback at the terminal is:
Traceback (most recent call last): File "csvToJSON.py", line 12, in <module> for row in reader: File "/usr/lib/python3.6/csv.py", line 112, in __next__ row = next(self.reader) File "/usr/lib/python3.6/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 11: invalid continuation byte The same error is produced with Python3– Omar
Dec 31 '18 at 21:20
|
show 2 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53983302%2fhow-to-fix-encoding-issue-in-a-python-script-for-spanish-alphabet%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53983302%2fhow-to-fix-encoding-issue-in-a-python-script-for-spanish-alphabet%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It seems to be in one of the ISO-8859-x encodings not in UTF-8.
– Klaus D.
Dec 31 '18 at 3:39
The Python 2
csvmodule used to have problems with UTF-8, not sure if they were ever fixed. Is there a good reason you're not using Python 3 which is the recommended and supported version of the language?– tripleee
Dec 31 '18 at 4:32
I see various problems with your code. For one, the
csvlibrary expects an open file (file handle), not a string. Also, the value ofcsvfileis of typeunicode, which thecsvlibrary can't handle in Python 2; it operates onstronly. Furthermore, the string"contraseña"should beu"contraseña". And it's unclear which line causes the decode error; please show the full traceback.– lenz
Dec 31 '18 at 9:47
And, as tripleee said, if you can, switch to Python 3. It will make your life much easier with respect to encodings.
– lenz
Dec 31 '18 at 9:48
@lenz The full traceback at the terminal is:
Traceback (most recent call last): File "csvToJSON.py", line 12, in <module> for row in reader: File "/usr/lib/python3.6/csv.py", line 112, in __next__ row = next(self.reader) File "/usr/lib/python3.6/codecs.py", line 321, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 11: invalid continuation byteThe same error is produced with Python3– Omar
Dec 31 '18 at 21:20