Special Character Encoding problem in Azure Data factory_V2

Multi tool use
Problem Statement:
Consider a value “für" is present in a column in csv file (Azure Blob) and when we copy that file using ADFV2 into Azure SQL table, column value now displays like this: "f�r".
These incorrect values are coming up in our business reports.
But at the same when do the same activity using SSIS and copy data to local database, it displays correct value “für".
Is there any way to handle these special characters in ADF_V2?

add a comment |
Problem Statement:
Consider a value “für" is present in a column in csv file (Azure Blob) and when we copy that file using ADFV2 into Azure SQL table, column value now displays like this: "f�r".
These incorrect values are coming up in our business reports.
But at the same when do the same activity using SSIS and copy data to local database, it displays correct value “für".
Is there any way to handle these special characters in ADF_V2?

Welcome to StackOverflow. We’d love to help you. To improve your chances of getting an answer, here are some tips: stackoverflow.com/help/how-to-ask
– Thilina Nakkawita
Dec 29 '18 at 16:39
Please set encodingName property in the source data set and destination data set as "UTF-8"
– Alberto Morillo
Dec 30 '18 at 20:34
Other encodings available docs.microsoft.com/en-us/dotnet/api/…
– Alberto Morillo
Dec 30 '18 at 20:41
add a comment |
Problem Statement:
Consider a value “für" is present in a column in csv file (Azure Blob) and when we copy that file using ADFV2 into Azure SQL table, column value now displays like this: "f�r".
These incorrect values are coming up in our business reports.
But at the same when do the same activity using SSIS and copy data to local database, it displays correct value “für".
Is there any way to handle these special characters in ADF_V2?

Problem Statement:
Consider a value “für" is present in a column in csv file (Azure Blob) and when we copy that file using ADFV2 into Azure SQL table, column value now displays like this: "f�r".
These incorrect values are coming up in our business reports.
But at the same when do the same activity using SSIS and copy data to local database, it displays correct value “für".
Is there any way to handle these special characters in ADF_V2?


edited Dec 31 '18 at 6:45
Ashish Bharadwaj
asked Dec 29 '18 at 15:50


Ashish BharadwajAshish Bharadwaj
11
11
Welcome to StackOverflow. We’d love to help you. To improve your chances of getting an answer, here are some tips: stackoverflow.com/help/how-to-ask
– Thilina Nakkawita
Dec 29 '18 at 16:39
Please set encodingName property in the source data set and destination data set as "UTF-8"
– Alberto Morillo
Dec 30 '18 at 20:34
Other encodings available docs.microsoft.com/en-us/dotnet/api/…
– Alberto Morillo
Dec 30 '18 at 20:41
add a comment |
Welcome to StackOverflow. We’d love to help you. To improve your chances of getting an answer, here are some tips: stackoverflow.com/help/how-to-ask
– Thilina Nakkawita
Dec 29 '18 at 16:39
Please set encodingName property in the source data set and destination data set as "UTF-8"
– Alberto Morillo
Dec 30 '18 at 20:34
Other encodings available docs.microsoft.com/en-us/dotnet/api/…
– Alberto Morillo
Dec 30 '18 at 20:41
Welcome to StackOverflow. We’d love to help you. To improve your chances of getting an answer, here are some tips: stackoverflow.com/help/how-to-ask
– Thilina Nakkawita
Dec 29 '18 at 16:39
Welcome to StackOverflow. We’d love to help you. To improve your chances of getting an answer, here are some tips: stackoverflow.com/help/how-to-ask
– Thilina Nakkawita
Dec 29 '18 at 16:39
Please set encodingName property in the source data set and destination data set as "UTF-8"
– Alberto Morillo
Dec 30 '18 at 20:34
Please set encodingName property in the source data set and destination data set as "UTF-8"
– Alberto Morillo
Dec 30 '18 at 20:34
Other encodings available docs.microsoft.com/en-us/dotnet/api/…
– Alberto Morillo
Dec 30 '18 at 20:41
Other encodings available docs.microsoft.com/en-us/dotnet/api/…
– Alberto Morillo
Dec 30 '18 at 20:41
add a comment |
1 Answer
1
active
oldest
votes
Try these steps:
- Open CSV file in notepad.
- Click on Save As.
- Save the CSV file by a name and select Encoding as UTF-8 in the Encoding drop down just left to the Save Button.
- Save the file.
- Re-try the ADF part and see for the results.
Regards,
Pratik
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53971008%2fspecial-character-encoding-problem-in-azure-data-factory-v2%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Try these steps:
- Open CSV file in notepad.
- Click on Save As.
- Save the CSV file by a name and select Encoding as UTF-8 in the Encoding drop down just left to the Save Button.
- Save the file.
- Re-try the ADF part and see for the results.
Regards,
Pratik
add a comment |
Try these steps:
- Open CSV file in notepad.
- Click on Save As.
- Save the CSV file by a name and select Encoding as UTF-8 in the Encoding drop down just left to the Save Button.
- Save the file.
- Re-try the ADF part and see for the results.
Regards,
Pratik
add a comment |
Try these steps:
- Open CSV file in notepad.
- Click on Save As.
- Save the CSV file by a name and select Encoding as UTF-8 in the Encoding drop down just left to the Save Button.
- Save the file.
- Re-try the ADF part and see for the results.
Regards,
Pratik
Try these steps:
- Open CSV file in notepad.
- Click on Save As.
- Save the CSV file by a name and select Encoding as UTF-8 in the Encoding drop down just left to the Save Button.
- Save the file.
- Re-try the ADF part and see for the results.
Regards,
Pratik
answered Dec 31 '18 at 9:19
Pratik SomaiyaPratik Somaiya
1085
1085
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53971008%2fspecial-character-encoding-problem-in-azure-data-factory-v2%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
rQTz 77cersa,cCv1h,60JcRNhiEeN,ABPwbdl1i,CIfpibyLUsQiDQVJMG,trfcu
Welcome to StackOverflow. We’d love to help you. To improve your chances of getting an answer, here are some tips: stackoverflow.com/help/how-to-ask
– Thilina Nakkawita
Dec 29 '18 at 16:39
Please set encodingName property in the source data set and destination data set as "UTF-8"
– Alberto Morillo
Dec 30 '18 at 20:34
Other encodings available docs.microsoft.com/en-us/dotnet/api/…
– Alberto Morillo
Dec 30 '18 at 20:41