Azure stream analytics query - how to set even timestamp based on a javascript udf function?

Multi tool use
Multi tool use












1














I am timestamping data stream input events by a property "TS" in the message. However before I timestamp it using TS, I want to ensure that TS is ISO8601 compliant. If TS is not ISO8601 ocmpliant, I want to use EventEnqueuedUtcTime which is the arrival time of the message as timestamp.



My query looks something like this



SELECT 
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY udf.getEventTimestamp(T)


Here udf.getEventTimestamp(T) returns the TS property in message(T) if it is ISO8601-compliant otherwise it will return EventEnqueuedUtcTime( arrival time of message in Iot Hub).



Running this script locally gives me the exception -
Error : Unexpected hosted function call



I also tried to use CASE construct to accomplish this



SELECT 
T.*
FROM
input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY
CASE
WHEN udf.isValid(T.TS) THEN T.TS
ELSE T.EventEnqueuedUtcTime
END


where udf.isValid(T.TS) returns true if the property TS is a valid ISO8601 compliant timestamp.



Again running this locally returns - Error : Unexpected hosted function call



As per Microsoft Azure docs, After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function



Does this mean that we cannot use udfs in TIMESTAMP BY and CASE constructs?
Can you suggest any workaround?










share|improve this question



























    1














    I am timestamping data stream input events by a property "TS" in the message. However before I timestamp it using TS, I want to ensure that TS is ISO8601 compliant. If TS is not ISO8601 ocmpliant, I want to use EventEnqueuedUtcTime which is the arrival time of the message as timestamp.



    My query looks something like this



    SELECT 
    T.*
    FROM
    input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY udf.getEventTimestamp(T)


    Here udf.getEventTimestamp(T) returns the TS property in message(T) if it is ISO8601-compliant otherwise it will return EventEnqueuedUtcTime( arrival time of message in Iot Hub).



    Running this script locally gives me the exception -
    Error : Unexpected hosted function call



    I also tried to use CASE construct to accomplish this



    SELECT 
    T.*
    FROM
    input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY
    CASE
    WHEN udf.isValid(T.TS) THEN T.TS
    ELSE T.EventEnqueuedUtcTime
    END


    where udf.isValid(T.TS) returns true if the property TS is a valid ISO8601 compliant timestamp.



    Again running this locally returns - Error : Unexpected hosted function call



    As per Microsoft Azure docs, After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function



    Does this mean that we cannot use udfs in TIMESTAMP BY and CASE constructs?
    Can you suggest any workaround?










    share|improve this question

























      1












      1








      1







      I am timestamping data stream input events by a property "TS" in the message. However before I timestamp it using TS, I want to ensure that TS is ISO8601 compliant. If TS is not ISO8601 ocmpliant, I want to use EventEnqueuedUtcTime which is the arrival time of the message as timestamp.



      My query looks something like this



      SELECT 
      T.*
      FROM
      input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY udf.getEventTimestamp(T)


      Here udf.getEventTimestamp(T) returns the TS property in message(T) if it is ISO8601-compliant otherwise it will return EventEnqueuedUtcTime( arrival time of message in Iot Hub).



      Running this script locally gives me the exception -
      Error : Unexpected hosted function call



      I also tried to use CASE construct to accomplish this



      SELECT 
      T.*
      FROM
      input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY
      CASE
      WHEN udf.isValid(T.TS) THEN T.TS
      ELSE T.EventEnqueuedUtcTime
      END


      where udf.isValid(T.TS) returns true if the property TS is a valid ISO8601 compliant timestamp.



      Again running this locally returns - Error : Unexpected hosted function call



      As per Microsoft Azure docs, After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function



      Does this mean that we cannot use udfs in TIMESTAMP BY and CASE constructs?
      Can you suggest any workaround?










      share|improve this question













      I am timestamping data stream input events by a property "TS" in the message. However before I timestamp it using TS, I want to ensure that TS is ISO8601 compliant. If TS is not ISO8601 ocmpliant, I want to use EventEnqueuedUtcTime which is the arrival time of the message as timestamp.



      My query looks something like this



      SELECT 
      T.*
      FROM
      input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY udf.getEventTimestamp(T)


      Here udf.getEventTimestamp(T) returns the TS property in message(T) if it is ISO8601-compliant otherwise it will return EventEnqueuedUtcTime( arrival time of message in Iot Hub).



      Running this script locally gives me the exception -
      Error : Unexpected hosted function call



      I also tried to use CASE construct to accomplish this



      SELECT 
      T.*
      FROM
      input TIMESTAMP BY PARTITION BY PartitionId TIMESTAMP BY
      CASE
      WHEN udf.isValid(T.TS) THEN T.TS
      ELSE T.EventEnqueuedUtcTime
      END


      where udf.isValid(T.TS) returns true if the property TS is a valid ISO8601 compliant timestamp.



      Again running this locally returns - Error : Unexpected hosted function call



      As per Microsoft Azure docs, After you add a JavaScript user-defined function to a job, you can use the function anywhere in the query, like a built-in scalar function



      Does this mean that we cannot use udfs in TIMESTAMP BY and CASE constructs?
      Can you suggest any workaround?







      azure-stream-analytics






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 6 hours ago









      user3740951

      88111




      88111





























          active

          oldest

          votes











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53941961%2fazure-stream-analytics-query-how-to-set-even-timestamp-based-on-a-javascript-u%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown






























          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53941961%2fazure-stream-analytics-query-how-to-set-even-timestamp-based-on-a-javascript-u%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          rxn93MAkA UuEoKqXOd2rPwp2N,Z,AV54 jB6f eNv3OSSiuScg54xfe 9evKTQGZ8z8zKowuOguY4QNdMAINnFlkgIqN0SDEMC9mNmx
          vHP,9wA1T7SL DOK2LA pTA8B,tvV1MAAXWOUWSo9zGV23Wu,Sn1vdF PX,Uk,6fl2 09ubsEs,w3GCfN 4

          Popular posts from this blog

          Monofisismo

          Angular Downloading a file using contenturl with Basic Authentication

          Olmecas