Home

Results 1 to 7 of 7

Thread: Automations fire multiple times when conditions met

Hybrid View

  1. #1
    csssl NNTP User

    Automations fire multiple times when conditions met


    We have automations set up as follows:

    Filter: Element at critical condition
    Element at major condition

    Actions: Run a predefined script.

    In the setup code for the predefined script:

    formula.log.error( "The element condition at automation trigger is
    ************** " + element.condition + " **************" )


    formula.trc clearly shows 4 automations are being triggered for the same
    element/condition. Why is this?

    2014-09-26 07:08:30,410 ERROR Script.Automation 'Ares', script 'Run a
    script' - The element condition at automation trigger is **************
    Critical **************
    2014-09-26 07:08:30,706 INFO Script.Automation 'Ares', script 'Run a
    script' - There were no incidents returned for the root cause incident
    check. Setting incident_exists to false. yes WE'RE gonna Create a
    Ticket!!!
    2014-09-26 07:08:35,433 ERROR Script.Automation 'Ares', script 'Run a
    script' - The element condition at automation trigger is **************
    Critical **************
    2014-09-26 07:08:35,636 INFO Script.Automation 'Ares', script 'Run a
    script' - The following tickets are open:
    2014-09-26 07:08:35,636 INFO Script.Automation 'Ares', script 'Run a
    script' - The Automation Evaluated the root cause of the Element Ares
    and did NOT create a ticket because it already exists in Service Now.
    2014-09-26 07:10:24,996 ERROR Script.Automation 'Ares', script 'Run a
    script' - The element condition at automation trigger is **************
    Critical **************
    2014-09-26 07:10:25,276 INFO Script.Automation 'Ares', script 'Run a
    script' - The following tickets are open:
    2014-09-26 07:10:25,276 INFO Script.Automation 'Ares', script 'Run a
    script' - The Automation Evaluated the root cause of the Element Ares
    and did NOT create a ticket because it already exists in Service Now.
    2014-09-26 07:10:25,276 ERROR Script.Automation 'Ares', script 'Run a
    script' - The element condition at automation trigger is **************
    Critical **************
    2014-09-26 07:10:25,479 INFO Script.Automation 'Ares', script 'Run a
    script' - The following tickets are open:
    2014-09-26 07:10:25,479 INFO Script.Automation 'Ares', script 'Run a
    script' - The Automation Evaluated the root cause of the Element Ares
    and did NOT create a ticket because it already exists in Service Now.

    Thanks.

    Steve


    --
    csssl
    ------------------------------------------------------------------------
    csssl's Profile: https://forums.netiq.com/member.php?userid=6773
    View this thread: https://forums.netiq.com/showthread.php?t=51881


  2. #2
    tisenberg NNTP User

    Re: Automations fire multiple times when conditions met


    I'm trying to remember the different use cases where this might happen
    but it has been awhile. We did have a bug in 4.7 (or a prior version)
    where automation was called accidently to many times but I think that
    was resolved a long time ago. My guess is that an underlying child
    element is getting an update and the parent element where the automation
    is, is going critical, then an update comes in and it is staying
    critical. For each of these, potentially automation is getting called
    based on your filter. You'll have to fire up the server side debugger
    and check, but I seem to remember a previous condition variable. So if
    previous and current are the same, just have your script bail out.

    Also, during your testing, if you have the thick client up on the
    Network view. Each time the element changes white it is getting some
    type of update at that level or lower (IE: new alarm, element property
    change, condition change, etc). That will cause things to be
    re-evaluated such as algorithms and potentially automations). My guess,
    if you are getting four automations, it is flipping white four times.

    Run debugger in context of the automation and see what variables are
    available within your automation script, see if there is something like
    previousCondition (or other variations).

    Also, if you do solve it, please update this forum thread.


    --
    tisenberg
    ------------------------------------------------------------------------
    tisenberg's Profile: https://forums.netiq.com/member.php?userid=1851
    View this thread: https://forums.netiq.com/showthread.php?t=51881


  3. #3
    csssl NNTP User

    Re: Automations fire multiple times when conditions met


    Thanks, Tobin.

    The event.priorCondition seems to be the variable. I was able to log
    this in an event this afternoon. The first trigger went from Minor to
    Critical. The next few triggers were from Critical to Critical each. We
    could certainly compare the current to prior state in the script, but is
    this the way it should be working in the first place?

    2014-10-02 14:48:12,713 ERROR Script.Automation 'Emory Commons', script
    'Run a script' - The element prior condition at automation trigger is
    ************** Minor **************
    2014-10-02 14:48:12,713 ERROR Script.Automation 'Emory Commons', script
    'Run a script' - The element condition at automation trigger is
    ************** Critical **************

    2014-10-02 14:48:16,675 ERROR Script.Automation 'Emory Commons', script
    'Run a script' - The element prior condition at automation trigger is
    ************** Critical **************
    2014-10-02 14:48:16,675 ERROR Script.Automation 'Emory Commons', script
    'Run a script' - The element condition at automation trigger is
    ************** Critical **************

    2014-10-02 14:48:17,752 ERROR Script.Automation 'Emory Commons', script
    'Run a script' - The element prior condition at automation trigger is
    ************** Critical **************
    2014-10-02 14:48:17,752 ERROR Script.Automation 'Emory Commons', script
    'Run a script' - The element condition at automation trigger is
    ************** Critical **************

    2014-10-02 14:48:19,749 ERROR Script.Automation 'Emory Commons', script
    'Run a script' - The element prior condition at automation trigger is
    ************** Critical **************
    2014-10-02 14:48:19,749 ERROR Script.Automation 'Emory Commons', script
    'Run a script' - The element condition at automation trigger is
    ************** Critical **************

    Steve


    --
    csssl
    ------------------------------------------------------------------------
    csssl's Profile: https://forums.netiq.com/member.php?userid=6773
    View this thread: https://forums.netiq.com/showthread.php?t=51881


  4. #4
    csssl NNTP User

    Re: Automations fire multiple times when conditions met


    I added the statement comparing event.priorCondition and
    element.condition. When it matches, the script ends. In order to test
    this, I set the element from OK to Critical. The automation triggered
    and the script ran as normal. Then I set the element to Critical again
    (i.e. from Critical to Critical). The automation triggered again, but
    the script bailed.


    --
    csssl
    ------------------------------------------------------------------------
    csssl's Profile: https://forums.netiq.com/member.php?userid=6773
    View this thread: https://forums.netiq.com/showthread.php?t=51881


  5. #5
    tisenberg NNTP User

    Re: Automations fire multiple times when conditions met


    I'm glad to hear it worked out. Per your other question of why does it
    work that way, I think it is to handle multiple use cases. In your
    case, you just want to know when it has gone Red/Critical. In other
    situations, maybe it is that the element needs to be red as well as some
    other property value set, or a specific amount of critical alarms, or,
    or, or. When there are "updates", this particular automation filter
    fires for each update when the element goes critical and/or is updated
    and remains critical.


    --
    tisenberg
    ------------------------------------------------------------------------
    tisenberg's Profile: https://forums.netiq.com/member.php?userid=1851
    View this thread: https://forums.netiq.com/showthread.php?t=51881


  6. #6
    csssl NNTP User

    Re: Automations fire multiple times when conditions met


    This works on an element condition, but is there any way to do a similar
    trick on an alarm? We have automations that trigger on "Critical alarm
    opened". The automation fires at least 3 times per alarm. The
    event.priorCondition does not work as it is always "undefined".


    --
    csssl
    ------------------------------------------------------------------------
    csssl's Profile: https://forums.netiq.com/member.php?userid=6773
    View this thread: https://forums.netiq.com/showthread.php?t=51881


  7. #7
    tisenberg NNTP User

    Re: Automations fire multiple times when conditions met


    I can only guess on your set up.

    a) you have the trigger for automation on any alarm opened and you are
    receiving multiple alarms.

    b) you have the trigger for an alarm updated and depending on the
    adapter, a single update may cause multiple updates (doubtful).

    c) you have an automation set at the (example) application, and multiple
    layers below the alarm comes in. The automation is trigger for the
    lowest child where the alarm hits, and each element between it and the
    parent that has the automation on it (most likely your situation).

    Assuming option "C", modify your script to check the element class and
    only fire for specific element types. Or... move the automation lower.
    Or... the automation could be placed in a different view that is less
    deep and provides the same general automation.

    Other than guessing, I'd would need more information to provide any
    other feedback. I hope this helps.


    --
    tisenberg
    ------------------------------------------------------------------------
    tisenberg's Profile: https://forums.netiq.com/member.php?userid=1851
    View this thread: https://forums.netiq.com/showthread.php?t=51881


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •