Azure stream analytics query - how to set even timestamp based on a javascript udf function?

Multi tool use
I am timestamping data stream input events by a property "TS" in the message. However before I timestamp it using TS, I want to ensure that TS is ISO8601 compliant. If TS is not ISO8601 ocmpliant, I want to use EventEnqueuedUtcTime which is the arrival time of the message as timestamp.
My query looks something like this
SELECT
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY udf.getEventTimestamp(T)
Here udf.getEventTimestamp(T)
returns the TS property in message(T) if it is ISO8601-compliant otherwise it will return EventEnqueuedUtcTime( arrival time of message in Iot Hub).
Running this script locally gives me the exception -
Error : Unexpected hosted function call
I also tried to use CASE
construct to accomplish this
SELECT
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY
CASE
WHEN udf.isValid(T.TS) THEN T.TS
ELSE T.EventEnqueuedUtcTime
END
where udf.isValid(T.TS)
returns true if the property TS is a valid ISO8601 compliant timestamp.
Again running this locally returns - Error : Unexpected hosted function call
As per Microsoft Azure docs, After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function
Does this mean that we cannot use udfs in TIMESTAMP BY and CASE constructs?
Can you suggest any workaround?
azure-stream-analytics
add a comment |
I am timestamping data stream input events by a property "TS" in the message. However before I timestamp it using TS, I want to ensure that TS is ISO8601 compliant. If TS is not ISO8601 ocmpliant, I want to use EventEnqueuedUtcTime which is the arrival time of the message as timestamp.
My query looks something like this
SELECT
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY udf.getEventTimestamp(T)
Here udf.getEventTimestamp(T)
returns the TS property in message(T) if it is ISO8601-compliant otherwise it will return EventEnqueuedUtcTime( arrival time of message in Iot Hub).
Running this script locally gives me the exception -
Error : Unexpected hosted function call
I also tried to use CASE
construct to accomplish this
SELECT
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY
CASE
WHEN udf.isValid(T.TS) THEN T.TS
ELSE T.EventEnqueuedUtcTime
END
where udf.isValid(T.TS)
returns true if the property TS is a valid ISO8601 compliant timestamp.
Again running this locally returns - Error : Unexpected hosted function call
As per Microsoft Azure docs, After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function
Does this mean that we cannot use udfs in TIMESTAMP BY and CASE constructs?
Can you suggest any workaround?
azure-stream-analytics
add a comment |
I am timestamping data stream input events by a property "TS" in the message. However before I timestamp it using TS, I want to ensure that TS is ISO8601 compliant. If TS is not ISO8601 ocmpliant, I want to use EventEnqueuedUtcTime which is the arrival time of the message as timestamp.
My query looks something like this
SELECT
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY udf.getEventTimestamp(T)
Here udf.getEventTimestamp(T)
returns the TS property in message(T) if it is ISO8601-compliant otherwise it will return EventEnqueuedUtcTime( arrival time of message in Iot Hub).
Running this script locally gives me the exception -
Error : Unexpected hosted function call
I also tried to use CASE
construct to accomplish this
SELECT
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY
CASE
WHEN udf.isValid(T.TS) THEN T.TS
ELSE T.EventEnqueuedUtcTime
END
where udf.isValid(T.TS)
returns true if the property TS is a valid ISO8601 compliant timestamp.
Again running this locally returns - Error : Unexpected hosted function call
As per Microsoft Azure docs, After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function
Does this mean that we cannot use udfs in TIMESTAMP BY and CASE constructs?
Can you suggest any workaround?
azure-stream-analytics
I am timestamping data stream input events by a property "TS" in the message. However before I timestamp it using TS, I want to ensure that TS is ISO8601 compliant. If TS is not ISO8601 ocmpliant, I want to use EventEnqueuedUtcTime which is the arrival time of the message as timestamp.
My query looks something like this
SELECT
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY udf.getEventTimestamp(T)
Here udf.getEventTimestamp(T)
returns the TS property in message(T) if it is ISO8601-compliant otherwise it will return EventEnqueuedUtcTime( arrival time of message in Iot Hub).
Running this script locally gives me the exception -
Error : Unexpected hosted function call
I also tried to use CASE
construct to accomplish this
SELECT
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY
CASE
WHEN udf.isValid(T.TS) THEN T.TS
ELSE T.EventEnqueuedUtcTime
END
where udf.isValid(T.TS)
returns true if the property TS is a valid ISO8601 compliant timestamp.
Again running this locally returns - Error : Unexpected hosted function call
As per Microsoft Azure docs, After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function
Does this mean that we cannot use udfs in TIMESTAMP BY and CASE constructs?
Can you suggest any workaround?
azure-stream-analytics
azure-stream-analytics
asked 6 hours ago
user3740951
88111
88111
add a comment |
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53941961%2fazure-stream-analytics-query-how-to-set-even-timestamp-based-on-a-javascript-u%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53941961%2fazure-stream-analytics-query-how-to-set-even-timestamp-based-on-a-javascript-u%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
rxn93MAkA UuEoKqXOd2rPwp2N,Z,AV54 jB6f eNv3OSSiuScg54xfe 9evKTQGZ8z8zKowuOguY4QNdMAINnFlkgIqN0SDEMC9mNmx