Mike Howells's Blog

Just another WordPress.com site

Archive for the ‘Administration’ Category

Diary of a Garmin BITS Job Gone Bad

Posted by mikehowells on February 27, 2014

It’s one of those e-mails that no one ever wants to receive…

Dear AT&T High Speed Internet Service Customer,

We want to remind you that your AT&T High Speed Internet service includes 150 gigabytes (GB) of data for each billing period..
You have exceeded 150 GB this billing period

What?!?

Of course, I believe it is an error. But, when I open my daily usage chart, I can clearly see this is no error:

Image

So, what in the world is downloading all of this data and why did it start on Wednesday the 19th?

I opened Microsoft’s Network Monitor and saw a multitude of requests to a Garmin subdomain caled nyc1.gdn.garminsource.net. It’s basically a CDN (Content Distribution Network) that Garmin utilizes to transfer high volume transactions such as map updates to its user base.

I have the Garmin Map Updater service installed. So, maybe it is downloading a new map for my Garmin device. But, 30 GB/day is far too excessive even for the largest of map updates.

I needed more tools at my disposal to determine what was happening. So, I downloaded and installed one of the best network bandwidth usage tools that I have ever come across. It’s called NetBalancer by SeriousBit. The NetBalancer desktop application allows you to view each process and how much bandwidth it is consuming. Once I opened the application, I could clearly see svchost.exe was consuming a rather large chunk of bandwidth.

Image

Now that the culprit was identified, how do I go about stopping it?

I suspected that Garmin utilized the BITS service. Utilizing the BITS service is a common practice for developers to use, which saves them the time from writing their own file transfer service. BITS stands for Background Intelligent Transfer Service. It’s an easily identifiable service, which can be stopped via the Services applet as shown below:

Image

As soon as I stopped the BITS service, the download immediately stopped and my bandwidth consumption returned to normal.

Another day passed and I re-opened NetBalancer and noticed that svchost.exe was consuming bandwidth again. I couldn’t believe it. The BITS service started itself up again. I even disabled the BITS service, which didn’t help. BITS would simply re-enable itself and then start itself. The activity started to feel malicious in nature.

It was at this point, I decided to uninstall everything Garmin on my desktop. For sure uninstalling my Garmin apps would fix it right?

Nope!

I uninstalled everything Garmin and the Garmin BITS jobs continued to consume my bandwidth with no end in sight. This had been going on for days. So, something must have gone terribly wrong with some Garmin code somewhere.

It was time to continue my investigation…

I found some BITS commands available via PowerShell.

The one I found to list the existing transfer jobs is this command get-Bitstransfer -allusers

Image

I then discovered there is a built-in command line utility called BITSADMIN that has all sorts of power!

I issued this command in an attempt to cancel all BITS jobs: BITSADMIN /reset /allusers

Image

No luck.

Of course, there is no reason given for the failure. But, after performing some research on canceling BITS jobs, it appears that you have to be logged in as the user who created the BITS job. So, how do you log in as NT AUTHORITY\SYSTEM? I actually blogged about this in 2011 in this blog article here: https://mikehowells.wordpress.com/2011/02/12/running-a-command-prompt-as-nt-authoritysystem/

Basically, you open a command prompt as administrator. Then, launch the SysInternals tool psexec.exe as SYSTEM and it will launch a command prompt as NT AUTHORITY\SYSTEM. I was feeling pretty confident that this would work.

Image

Nope. It failed miserably. The error indicates that the request failed because the user (i.e. SYSTEM) has not logged on to the network. This was a fatal blow because the NT AUTHORITY\SYSTEM account is not designed to gain access to the network. This is usually reserved for the NETWORK SERVICE account.

So, I decided to fire-up my good friend ProcMon. ProcMon, or Process Monitor, is another brilliantly written tool that is part of the SysInternals Suite. After launching ProcMon, I included only the process svchost.exe. I could then clearly see the folder that svchost.exe was accessing, which was: C:\ProgramData\Garmin\Core Update Service\MAP-NA-2014-40

It was clear to me that the Garmin uninstaller did not do a good job of cleaning-up after itself at all.

At this point, I had two options moving forward:

Option 1) Use the NetBalancer tool to limit the download/upload rate for svchost.exe. This was not preferable as many things use svchost.exe and it would have unintended consequences.

Option 2) Delete the C:\ProgramData\Garmin\ folder.

I opted for Option 2.

This stopped the BITS job and put it into an error state. At least it wasn’t downloading.

I now have two strikes against me from AT&T. If I go over my 150 GB threshold one more time, I will be charged $10 for each 50 GB over my limit. Why does AT&T have such a low threshold for its DSL user base? It’s basically AT&T’s way to force you into their U-Verse service. Even their U-Verse service only has a 250 GB/month limit, although I hear it’s not enforced.

I still have two remaining Garmin jobs that are sitting in a suspended state.

If anyone has any ideas how to delete these stale jobs I would like to hear from you!

Posted in Administration | 1 Comment »

Running a Command Prompt as NT AUTHORITY\SYSTEM

Posted by mikehowells on February 12, 2011

I recently ran into a situation where I was using the SysInternals tool ProcDump to write a dump file to be examined for a memory leak.

The problem started when trying to run ProcDump against the process oracle.exe. The error message was “Access denied.”

I was an administrator on the server so how could I become more powerful than an administrator?

The answer comes in the form of opening a command prompt as NT AUTHORITY\SYSTEM, which will then grant us the authority to access the oracle.exe process to create a dump file.

The first step is to download the Sysinternals tool PsExec from the below URL:

http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx

Extract PsTools.zip to a folder on your hard disk.

Launch a command prompt as administrator (right-click the command prompt shortcut):

In the command prompt navigate to the folder containing the PsTools.zip extracted data.

We will now launch PsExec.exe with the -i and -s switches to launch the program interactively using Local System.

psexec.exe -i -s %SystemRoot%\system32\cmd.exe

Type whoami at the newly opened command prompt and you will see that you are now running as NT AUTHORITY\SYSTEM:

You can now execute ProcDump against the process that you were previously denied access to and complete your work.

Note: If your system does not have whoami.exe, you can typically find this program as a separate download via the resource kit or support tools appropriate to your Microsoft operating system.

Posted in Administration | 7 Comments »