Network Automation Software We are currently migrating this forum
over to our HelpSystems domain. Please
post all new threads in our new
HelpSystems Community Portal.
Post to the HelpSystems Forum
You are not currently logged on. You must be logged on in order to post. Log on
Or Create a new account
AutoMate Discussion
Decrease font size
Increase font size
Topic Title: File System Trigger not working as expected
Topic Summary: When Using the File System Trigger my Task runs multiple times simutaneously
Created On: 03/21/2012 05:53 AM
Status: Read Only
Linear : Threading : Single : Branch
Search Topic Search Topic
Topic Tools Topic Tools
View similar topics View similar topics
View topic in raw text format. Print this topic.
 03/21/2012 05:53 AM
User is offline View Users Profile Print this message

Author Icon
SharneathaSheppard
Student (5-19 posts)

Posts: 10
Joined: 01/18/2012

When Using the File System Trigger my Task runs multiple times simutaneously,
We are running 9.0.0.25 on a Server 2008 R2 server.

Here the issue I am experiencing;

I have a Task named MOVE R20s via FTP (WEB) that is triggered by files being added to a specified folder. There can be 5-6 file hitting the folder at the same time.

The Task then copies these 5-6 files to another location and then moves these file to another location. The problem I am experiencing is the Task appears to be running 5-6 times simultaneously. Note, the files appears to have been moved as expected however I am also receiving Task Error failures which contains the following message.

Copy File Error: "The process cannot access the file because it is being used by another process.
Error in moving folder: "Could not find file XXXXXXXXXXXXX

Another task ZIP, RENAME and MOVE R20s (WEB SERVICE) is then triggered when the files from are copied to a specified folder. There same 5-6 files are hitting the folder at the same time.

This task loops through the the specified folder and renames the 5-6 files, then compresses thesse files to the destination folder; and then these files are deleted. The 5-6 files are moved, renamed, and zipped as expected however both tasks ends with failures, I am also receiving Task Error failures which contains the following message.

No files compressed.
No files found to delete.

This does is not a problem when running the task manually or scheduling the task to run at a specific time. Both Task were migrated from Automate 7, i did not have this problem while running these taks in Automate.

Additional notes;
We have these tasks to run as follows;

Priority - Run task if the number of running task instances of this task is below the threshold (1).
Task Isolation - Default

I there some addtional steps i need to consider when running these task on BPA Server 9? Or is this and known issue for BPA Server 9?

 Category Survey



Task Steps: (Right click, Select All, Copy to copy to the clipboard. Steps can then be pasted into the AutoMate Task Builder)
 03/21/2012 06:38 AM
User is offline View Users Profile Print this message

Author Icon
Leonard Amabile
HelpSystems

Posts: 5454
Joined: 05/15/2007

Hi,

I'm assuming you have the 'file added' and/or 'file modified' selected in your file watcher trigger correct? The problem I'm seeing is that for every file dropped in the folder, a task is triggered. If the task is not intended to only work with the file the caused the triggering event then you will run into conflicts. From what I see in your code, you are using wildcards and the task accounts for every file in the folder. What is occurring is that the other queued tasks that triggered will no longer find the files present because the first triggered task handled all of the files already. To alleviate the issue, you either need to use the schedule trigger as you mentioned in your post or have the task only manipulate the file that triggered it.




 Category Survey
AutoMate BPA Server 9 version:
Windows version:


-------------------------

Leonard Amabile | Director, Cross Platform Support
AutoMate | InterMapper | Skybot | Divisions of HelpSystems
T: +1.213.738.1700 | F: +1.213.738.7665
 03/21/2012 07:26 AM
User is offline View Users Profile Print this message

Author Icon
SharneathaSheppard
Student (5-19 posts)

Posts: 10
Joined: 01/18/2012

Thanks for your reply.


Yes, the File Condition prerequisite is set to 'File added to folder'. The task in not intended to only work with the file that caused the triggering event therefore I will continue to use the schedule trigger for these task. I think this is a bummer since this was a nonissue with Automate 7.
I will keep this in mind when using the file system trigger for newly created task.

Thanks again!

Sharneatha

 Category Survey
 03/21/2012 08:53 AM
User is offline View Users Profile Print this message

Author Icon
SharneathaSheppard
Student (5-19 posts)

Posts: 10
Joined: 01/18/2012

One more thought i'd like to added we have set the following Priority to;

Run task if the number of running task instances of this task is below the threshold (1).
If the condition above is not initially met:
Do not run task

Therefore we would not expect the Task to be triggered multiple times by the addition of multiple files. We think there is a bug with the Priority function in BPA Server 9 because these settings worked fine in Automate 7.

Sharneatha



 Category Survey
 03/22/2012 12:31 PM
User is offline View Users Profile Print this message

Author Icon
Leonard Amabile
HelpSystems

Posts: 5454
Joined: 05/15/2007

Hi Sharneatha.

Is it possible to export the AM7 task and attach the .aml file to your reply so that we can review the properties and contents? Or create a smaller sample task in AM7 and attach that file instead? We need a reproducible case to test internally for verification.





 Category Survey
AutoMate BPA Server 9 version:
Windows version:


-------------------------

Leonard Amabile | Director, Cross Platform Support
AutoMate | InterMapper | Skybot | Divisions of HelpSystems
T: +1.213.738.1700 | F: +1.213.738.7665
 03/27/2012 12:32 PM
User is offline View Users Profile Print this message

Author Icon
SharneathaSheppard
Student (5-19 posts)

Posts: 10
Joined: 01/18/2012

Sorry for the delay in getting back to you.

Attached are the aml files for your review.



 Category Survey
 03/28/2012 02:32 PM
User is offline View Users Profile Print this message

Author Icon
Ricardo Castaneda
HelpSystems

Posts: 3424
Joined: 01/11/2008

Hi Sharneatha,

I do not see an attachment posted. Can you please attach the file again?

 Category Survey


-------------------------

Ricardo Castaneda | Principal Support Analyst
AutoMate | Division of HelpSystems
HelpSystems, LLC | T: +1.213.738.1700
 03/29/2012 10:57 AM
User is offline View Users Profile Print this message

Author Icon
SharneathaSheppard
Student (5-19 posts)

Posts: 10
Joined: 01/18/2012

Hello,

Sample Task attached again.

Thanks,

 Category Survey
 04/01/2012 11:01 AM
User is offline View Users Profile Print this message

Author Icon
ChristofferJönsson
Student (5-19 posts)

Posts: 6
Joined: 03/17/2012

As mentioned in the posts above...

I have run into this problem as well and from my experience the only way to run tasks like this is to have logic that handles only one file at a time and then build a loop that processes all files one by one. Otherwise you will get all kinds of problems. Also you need to ensure that the task does not trigger multiple instances because if it does two jobs from the same task might process the same files. I always lock my jobs to single instances and single file processing, alternatively be _very_ sure that your jobs will finish executing between the trigger timer intervals (if that is what you are using).



 Category Survey


-------------------------
<html />
 03/05/2013 06:48 AM
User is offline View Users Profile Print this message

Author Icon
JenniferAnderson
Apprentice (20-49)

Posts: 24
Joined: 12/12/2011

I know it has been some time since this topic was discussed but wanted to know if anyone ever responded about this since it Muliple files did work properly in Automate 6 and 7, and now even with the Threshold set it doesn't work which causes us much annoyance with failed tasks and not all the files always behaving as they should. So lots of manual cleanup from a tool meant for Automation. If it hasn't been fixed is it considered a bug and is it on the list to fix? Even if the Threshold piece could be fixed that would be a huge help.

 Category Survey
 03/05/2013 06:56 AM
User is offline View Users Profile Print this message

Author Icon
Leonard Amabile
HelpSystems

Posts: 5454
Joined: 05/15/2007

Jennifer,

What is the exact issue you are having with multiple files. The file trigger itself has functioned the same since AM6. We do have a lot of users that, within their task, are moving files using a wildcard. So if you drop 2 files in the folder, 2 tasks will trigger but the 1st task will handle all of the files before the 2nd triggered task gets to the files. This is a conflict with how the tasks are designed. In order to prevent conflicts, file manipulation based on a file trigger should reference %amtrigger.filename%. This ensures that every task that is triggered by the file trigger will only handle the triggering file.




 Category Survey
AutoMate BPA Server 9 version:
Windows version:


-------------------------

Leonard Amabile | Director, Cross Platform Support
AutoMate | InterMapper | Skybot | Divisions of HelpSystems
T: +1.213.738.1700 | F: +1.213.738.7665
 03/05/2013 07:09 AM
User is offline View Users Profile Print this message

Author Icon
JenniferAnderson
Apprentice (20-49)

Posts: 24
Joined: 12/12/2011

I think there has been enough responses to how multiple files are triggered in 9 vs, 6 and 7 to warrant this as an issue. Even if you don't use the %amtrigger.filename%. so I think you are saying there is no issue there and it will never work in like it did in 6 or 7. Ok fine how about when you set the Threshold to have only one of a particular task run at a time? Is this a known issue? Since when we have it set to 1, it will still kick off as many tasks as there are files to process for a given trigger folder.

 Category Survey
 03/05/2013 07:29 AM
User is offline View Users Profile Print this message

Author Icon
Leonard Amabile
HelpSystems

Posts: 5454
Joined: 05/15/2007

Jennifer,

You are missing a portion of your priority setting that is very important to how you are troubleshooting the issue. This thread is for BPA so I'm assuming that is what you are troubleshooting correct? Please provide the exact priority settings you have set for the task. Do you have the priorities set to hold task until condition is met? If so, then if you are not using %amtrigger.filename% then all subsequent "held" tasks will fail because the initial task processed all of the files. Based on those priorities, if you drop 5 files into the monitored folder, then 5 tasks will trigger. One task will execute while the other 4 wait in a queue. If the initial task processes all 5 files then the next 4 queued tasks will fail once they execute.



 Category Survey
AutoMate BPA Server 9 version:
Windows version:


-------------------------

Leonard Amabile | Director, Cross Platform Support
AutoMate | InterMapper | Skybot | Divisions of HelpSystems
T: +1.213.738.1700 | F: +1.213.738.7665
 02/16/2016 07:04 AM
User is offline View Users Profile Print this message

Author Icon
StuartNelson
Student (5-19 posts)

Posts: 10
Joined: 08/07/2015

Was this ever sorted? I am having a similar problem with multiple WorkFlows triggering from a single Schedule trigger.



 Category Survey
AutoMate BPA Server 9 version: 9.0.4
Windows version: Windows Server 2003
 02/16/2016 07:07 AM
User is offline View Users Profile Print this message

Author Icon
StuartNelson
Student (5-19 posts)

Posts: 10
Joined: 08/07/2015

See:

http://forums.networkautomation.com/forum/messageview.cfm?catid=51&threadid=13826&highlight_key=y&keyword1=priorities

 



 Category Survey
AutoMate BPA Server 9 version: 9.0.4
Windows version: Windows Server 2003
 02/16/2016 07:21 AM
User is offline View Users Profile Print this message

Author Icon
Leonard Amabile
HelpSystems

Posts: 5454
Joined: 05/15/2007

Originally posted by: StuartNelson

Was this ever sorted? I am having a similar problem with multiple WorkFlows triggering from a single Schedule trigger.



Hi Stuart,

This thread is based on the File System trigger and not the Schedule trigger. We will troubleshoot your issue on the other thread.



 Category Survey
AutoMate BPA Server 9 version:
Windows version:


-------------------------

Leonard Amabile | Director, Cross Platform Support
AutoMate | InterMapper | Skybot | Divisions of HelpSystems
T: +1.213.738.1700 | F: +1.213.738.7665
Statistics
18258 users are registered to the AutoMate Discussion forum.
There are currently 0 users logged in.
The most users ever online was 5551 on 01/08/2018 at 11:11 AM.
There are currently 1514 guests browsing this forum, which makes a total of 1514 users using this forum.

FuseTalk Enterprise Edition v4.0 - © 1999-2020 FuseTalk Inc. All rights reserved.

Sitemap Network Automation Software Blog