Network Automation Software We are currently migrating this forum
over to our HelpSystems domain. Please
post all new threads in our new
HelpSystems Community Portal.
Post to the HelpSystems Forum
You are not currently logged on. You must be logged on in order to post. Log on
Or Create a new account
AutoMate Discussion
Decrease font size
Increase font size
Topic Title: SMC sees Agent as offline but the Agent service on the agent machine is still running (#3630)
Topic Summary:
Created On: 02/28/2012 03:17 PM
Status: Read Only
Linear : Threading : Single : Branch
Search Topic Search Topic
Topic Tools Topic Tools
View similar topics View similar topics
View topic in raw text format. Print this topic.
 02/28/2012 03:17 PM
User is offline View Users Profile Print this message

Author Icon
ArnoldHari
Student (5-19 posts)

Posts: 19
Joined: 12/20/2010

We've just upgraded to v8.0.9 of Automate BPA and there's been at least 3 seperate occassions where the SMC displayed Agents offline but when I login to those Agent servers, the Automate BPA Agent service is still in "Started" status, there are no relevant messages in the Event Viewer, and when I try to stop the service, I get an error saying the stop request failed (or something to that effect). To stop the service, I had to kill the process using Task Manager. Everything was back to normal after I restarted the Agent service. What would cause SMC to see an Agent as 'offline'.
I also submitted a ticket with NA asking if there's a way to setup a notification to go out when an Agent goes 'offline' and I was told there is no way to do that at present. Is it just me or should this have been at the top of the features list when Automate BPA Server was being designed?

 Category Survey


Edited: 02/29/2012 at 01:37 PM by AutoMate Discussion Moderator
 02/29/2012 01:36 PM
User is offline View Users Profile Print this message

Author Icon
Leonard Amabile
HelpSystems

Posts: 5454
Joined: 05/15/2007

Hi Arnold,

I know it's an obvious question but I have to ask anyway; did you upgrade all of your BPA components to 8.0.9.0? The issue that you are experiencing was addressed in 8.0.9.0 so we need to make sure that all components are of the same version.

Also, your suggestion for agent offline notification has been added into our tracking database.





 Category Survey
AutoMate BPA Server 8 version:
Windows version:


-------------------------

Leonard Amabile | Director, Cross Platform Support
AutoMate | InterMapper | Skybot | Divisions of HelpSystems
T: +1.213.738.1700 | F: +1.213.738.7665
 03/02/2012 01:57 PM
User is offline View Users Profile Print this message

Author Icon
ArnoldHari
Student (5-19 posts)

Posts: 19
Joined: 12/20/2010

Yes, the BPA Server and all Agent machines are on 8.0.9. This actually just happened again yesterday when I bounced the BPA Server. After it came back up, all but 3 Agents came right back online. The other 3 I had to kill the BPA_TS Process via Task Manager because stopping it via the Services console failed.



 Category Survey
 03/02/2012 08:50 PM
User is offline View Users Profile Print this message

Author Icon
JeanDelfosse
Supreme Guru (1000+)

Posts: 8883
Joined: 03/30/2005

In my opinion, the notification when an agent turns offline is a nice feature, but it should be an option per agent, because there are agents "on-the-road" that can be turned on and off without the need for a notification.

However there is also the need, unfulfilled by now, to have a trigger available in the server to fire when an agent turns on or off, which may differ from the one Arnold suggested.

 Category Survey
AutoMate BPA Server 8 version:
Windows version:


-------------------------
Jean Delfosse - Automate Beta Tester
Win7 Pro 64 bits -Win XP Pro SP3 32 bits - AM8, AM7, BPA10, BPA9, BPA8, BPA7, AM6 Enterprise, AM4
 03/05/2012 02:08 PM
User is offline View Users Profile Print this message

Author Icon
Leonard Amabile
HelpSystems

Posts: 5454
Joined: 05/15/2007

Hi Arnold,

What are the operating systems for all components involved? Are there any references in Event Viewer that may shed more light on the disconnects?






 Category Survey
AutoMate BPA Server 8 version:
Windows version:


-------------------------

Leonard Amabile | Director, Cross Platform Support
AutoMate | InterMapper | Skybot | Divisions of HelpSystems
T: +1.213.738.1700 | F: +1.213.738.7665
 03/26/2012 04:01 PM
User is offline View Users Profile Print this message

Author Icon
ArnoldHari
Student (5-19 posts)

Posts: 19
Joined: 12/20/2010

Hi Leonard,
Sorry I've been away so long. The agents are all running Windows Server 2003 and no, I didn't see anything recorded in the Windows Event log related to the Agent going offline. As I indicated, The Agent service does not stop on the Agent machine, but somehow the BPA SMC sees it as offline.

A variation of this problem is what I encountered last Friday. I fired off a workflow to run on an Agent Group that has 9 agents. The workflow appeared to run on all agents except two. When I logged on via Remote Desktop to those machines, it looked like the workflow kicked off because I can see a BPATASK process in Task Manager; however, it was either not doing anything or it was stuck a Task step.

I'm sorry to sound dense but in Automate 6, I can 'Show Running Tasks' but I can't seem to find this feature in Automate BPA 8. Did I completely miss the boat?

 Category Survey
 03/28/2012 11:56 AM
User is offline View Users Profile Print this message

Author Icon
Liz Casale
AUTOMATE EMPLOYEE

Posts: 940
Joined: 05/23/2011

Hi Arnold,

BPA does not have the same "Show Running Tasks" option. However, if you have the SMC open you can view the running tasks by reviewing the "Execution Status" tab.

Are there any patterns to the disconnects? In the SMC --> Options --> Licenses, how many Client Access Licenses do you have?

 Category Survey


-------------------------
Liz Casale
Technical Support Representative
(213) 738-6966
Network Automation, Inc.

--------------------------------------------------------------------------------

AM5 5.5.5.1; AM6 6.2.8.0; AM7 7.1.3.0; AM8 8.0.9.0; AM9 9.0.3.4
BPA7 7.1.3.0; BPA8 8.0.9.0; BPA9 9.0.3.4
 03/28/2012 04:20 PM
User is offline View Users Profile Print this message

Author Icon
ArnoldHari
Student (5-19 posts)

Posts: 19
Joined: 12/20/2010

Yes, I know about the Execution Status tab in the SMC but that only shows me status for Workflows or Tasks I executed from the SMC in the current session. In Automate 6, "Show Running Tasks" showed me all Tasks currently running on that particular machine and what step it's in. Not having this feature in BPA 8 is a huge disservice and makes troubleshooting in real-time almost impossible because there's no way to tell, at a glance, what's currently running on a particular Agent and what step it's on, especially if I suspect that it's hung up. Looking at the Agent's Task Manager doesn't help because all you see is a process called "BPATask" for every task that's currently running, but you have no idea which tasks they are. As it is, to stop a Task that we suspect is hanging means to stop all "BPATask" processes, then run it in Debug mode. Not very efficient especially if we have hundreds of Tasks running and some that may be scheduled to run every 5 minutes or worse, every minute.

We've had Automate BPA Server 8 for almost a year but have only recently started to migrate our jobs from Automate 6. I feel like upgrading is, at times, 2 steps forward and then we hit a wall and either have to step back or move sideways until we figure out a workaround. Call me crazy but for a software product that claims to be enterprise-grade, simple features like "Show Running Tasks" should not be overlooked.

 Category Survey
 03/28/2012 04:22 PM
User is offline View Users Profile Print this message

Author Icon
ArnoldHari
Student (5-19 posts)

Posts: 19
Joined: 12/20/2010

By the way, Liz, in response to your question, we have 51 CALs and only 49 Agents deployed, at present.

 Category Survey
 03/28/2012 08:35 PM
User is offline View Users Profile Print this message

Author Icon
Hamid Noorbakhsh
HelpSystems

Posts: 454
Joined: 07/29/2008

Originally posted by: ArnoldHari

Yes, I know about the Execution Status tab in the SMC but that only shows me status for Workflows or Tasks I executed from the SMC in the current session.



Arnold, please change the preferences to see all events as shown. Depending on the license type you have you might not have some of the setting as seen below.
Click on Options/Preferences/Logging Output and check to see all entries to see everything. This option is off by default to reduce 'noise' in the panel.

Also another option is to click on Dashboard. Once activated, the default view provided has gadgets that show running Tasks and Workflows that you can check out.

Hope this helps.

 Category Survey


-------------------------
Thank you for choosing AutoMate

Hamid Noorbakhsh | Senior Software Engineer
AutoMate | A Division of HelpSystems
HelpSystems, LLC | T: +1.213.738.1700
 03/29/2012 01:38 PM
User is offline View Users Profile Print this message

Author Icon
ArnoldHari
Student (5-19 posts)

Posts: 19
Joined: 12/20/2010

Thanks, Hamid. Too bad it can only show you 'Output' and 'Execution Status' for jobs that kickoff after logging on to SMC but not those that were already running. Anyway, this should be good enough for most jobs.

 Category Survey
 03/29/2012 01:45 PM
User is offline View Users Profile Print this message

Author Icon
ArnoldHari
Student (5-19 posts)

Posts: 19
Joined: 12/20/2010

Going back to the original issue, we're still seeing instances where SMC shows an Agent as 'Offline' yet on the Agent machine, the BPA Agent service is still running and there are no Windows Events recorded on the BPA Server or the Agent machine. Not having the ability for the SMC to send out notification when it sees 'Offline' agents is a major gap for us because if this happens after-hours, we could have jobs that have not been running overnight and not know about it until we check SMC the next morning.

 Category Survey
 03/29/2012 01:47 PM
User is offline View Users Profile Print this message

Author Icon
Hamid Noorbakhsh
HelpSystems

Posts: 454
Joined: 07/29/2008

Arnold,
The currently running Tasks/Workflows should still show in the Execution Panel, so anytime you launch SMC, if there is anything running, it should show.
Another choice is that you can stop 'Auto Refresh' by right clicking on the panel and checking it off and refresh anytime manually and get anything that is running.


Edit - Agent going offline is obviously detected, however this acting as a trigger to allow other functionality has not been implemented yet and has been brought up for future implementation.

 Category Survey


-------------------------
Thank you for choosing AutoMate

Hamid Noorbakhsh | Senior Software Engineer
AutoMate | A Division of HelpSystems
HelpSystems, LLC | T: +1.213.738.1700

Edited: 03/29/2012 at 02:00 PM by Hamid Noorbakhsh
 03/29/2012 06:38 PM
User is offline View Users Profile Print this message

Author Icon
ArnoldHari
Student (5-19 posts)

Posts: 19
Joined: 12/20/2010

Hamid, the issue I raised is not whether the SMC detects the Agent going offline but how is it detecting this. The Agent machine thinks everything's fine because the BPA Agent service is still in "Started" mode, yet when this situation occurs jobs won't run on that agent and we are unable to stop the BPA Agent service normally from the Services snap-in. I've had to kill the BPA_TS process from Task Manager so that I can restart it.

 Category Survey
 03/29/2012 09:04 PM
User is offline View Users Profile Print this message

Author Icon
JeanDelfosse
Supreme Guru (1000+)

Posts: 8883
Joined: 03/30/2005

There is indeed a need for a trigger detecting agents going offline or online, in businesses with roaming agents, like people on the road.
And also of course in situations like maintenance or system failure as mentioned in this thread.

 Category Survey
AutoMate BPA Server 8 version:
Windows version:


-------------------------
Jean Delfosse - Automate Beta Tester
Win7 Pro 64 bits -Win XP Pro SP3 32 bits - AM8, AM7, BPA10, BPA9, BPA8, BPA7, AM6 Enterprise, AM4
Statistics
18258 users are registered to the AutoMate Discussion forum.
There are currently 0 users logged in.
The most users ever online was 5551 on 01/08/2018 at 11:11 AM.
There are currently 1654 guests browsing this forum, which makes a total of 1654 users using this forum.

FuseTalk Enterprise Edition v4.0 - © 1999-2020 FuseTalk Inc. All rights reserved.

Sitemap Network Automation Software Blog