How to run Docker Windows Containers with McAfee Endpoint Security

McAfee Endpoint Security and I have a love/hate relationship in that I hate it when it gets in my way and love it when it’s not installed. In general, I appreciate security and security research, but recently I had been trying out (or attempting to try out) Docker and Kubernetes for a project I’m working on. It’s a .NET 4.6 web application, and as such, requires Windows Server Core (as opposed to the much lighterweight, new Windows Nano Server or a Linux-based container).  The fact that we now have an option to effectively containerize any application, including a windows application, is incredible. So yesterday, I decided to try and set it up. Here’s how it went.

What OS to target with .NET Containers

The only potential downside that I had heard about Windows Server Core containers were that they were pretty big. When I say big I mean like real big. 10GB big.

Ok, fine, that’s…that’s not good but there’s no other way to run full framework .NET apps without it a WindowsServerCore container, so I can get past that.

Installing Docker and gotchas

Installing Docker on Windows with McAfee is pretty straightforward. This was the first time I’d installed it in a couple years, so I was pleasantly surprised when the entire thing was pretty seamless. The wizard created the necessary docker-users security group and added me to it and has a pretty nice interface now. There were a few initial setup steps that were unclear I had to lookup, though:

  1. You need to add your user to the Hyper-V Administrators group. For whatever reason, the Docker for Windows installation created the docker-users group and added me there, but didn’t to the existing Hyper-V Administrators group. 🤷

  2. When building a docker container via the command line interface (Docker CLI), the login information is different from what you use to log into the website.  The website accepts you username OR your email address (I use my email address), but the CLI will only accept your username, and not the email address.  It also doesn’t tell you why the login fails, so just get in a habit of using your username.

Issues building a docker image with McAfee Endpoint Security installed

Everything seemed to be going as well as I could expect until it came time to run docker build on my dockerfile and get everything set up.  It seemed to be pulling things from the registry OK, but then I was met with this error message:

PS C:\WINDOWS\system32> docker pull microsoft/windowsservercore
Using default tag: latest
latest: Pulling from microsoft/windowsservercore
9c7f9c7d9bc2: Extracting [==================================================>] 3.738 GB/3.738 GB
d33fff6043a1: Download complete
failed to register layer: rename C:\ProgramData\Docker\image\windowsfilter\layerdb\tmp\write-set-925881297 C:\ProgramDat
a\Docker\image\windowsfilter\layerdb\sha256\3fd27ecef6a323f5ea7f3fde1f7b87a2dbfb1afa797f88fd7d20e8dbdc856f67: Access is
denied.

Ok, so let’s look at what’s happening here.  It’s saying it can’t register a layer because access is denied on my computer’s file system…but I’m administrator! What could possibly be the issue? Then I remember. McAfee. A quick Google search showed a lot of other folks having the same kind of issue, and ends up that McAfee doesn’t even support Windows containers. Yea, that’s right.

McAfee Endpoint Security Does Not Support Windows-based Docker Containers!

After tons of research, the answer from McAfee itself (according to a knowledge base article), is that McAfee Endpoint Security does not support Windows Docker containers! It seems the main issues were:

  • DNS does not resolve
  • Performance issues with containers
  • Slowness in opening containers
  • Firewall/NAT issues

To take a look for yourself, here’s the original KB article on Windows Docker containers not being supported by McAfee.

How can I run Windows Docker Containers on Windows if McAfee doesn’t support it?

There are actually a few workarounds that have been successful for me and my team:

Virtual Machines

First, you can use the docker tools inside a separate “real” virtual machine (using VMWare, VirtualBox, etc) and actually perform the work there. This gets you around McAfee being on the VM directly, but also has a few drawbacks. It will slow your workflow down as you’ll have the VM to jump through from your local machine to the docker tools and images you may be running. This is not ideal, but might give you some runway until McAfee can add support.

.NET Core!

McAfee says that Linux-based containers are OK. This means if you’re using a language or tools that require Windows containers, you may be able to move to something that doesn’t require them.

I know this isn’t much help, particularly if you have a large application already that you’re moving to Docker, but can be an option if you’re just starting out or if you are able to adjust your stack enough to be able to drop the Windows requirement.

Remove McAfee?

Move away from McAfee if Windows Docker containers are a hard requirement for you and no other options work well for your team.

The bottom line

Basically, McAfee is really tough to work around as a developer, and Windows Docker containers are no exception. You have a limited options in the meantime, but I feel like the best recommendation is to not waste your time with workarounds and wait it out. McAfee should have an update at some point in the future to fix the issue. If you can’t afford to wait and the workarounds aren’t options, remember that making your voice heard on Twitter and Email and letting McAfee know this is important will go a long way in having them increase this as a priority.

How to Permanently Fix “Could not copy “C:\MyProject\MyDLL.dll” to “C:\MyProject\bin\Debug\MyDLL.dll”.

Recently I had an issue where when I would try to run and debug a webapp inside Visual Studio 2017, I would get an error where the .dlls couldn’t be copied out to the bin directory, which meant I couldn’t actually run the app at all! I would keep getting this error:

[blockquote]”Could not copy “C:\MyProject\MyDLL.dll” to “C:\MyProject\bin\Debug\MyDLL.dll”[/blockquote]

This error is usually caused by a lock on the target file, preventing it from being deleted.

There was only one problem – I had restarted Visual Studio and it was still happening.

I even restarted my computer, which fixed the problem once, but subsequent runs brought the error back.

The Root Cause

The actual root cause ended up that I actually had multiple projects set up as my startup project, but ONE of the projects wasn’t set up to debug, while the other was.

When you set the configuration in Visual Studio for a project to run without debugging, it actually doesn’t give the app the hooks that Visual Studio needs in order to close the app once you select “Stop Debugging”.  This causes you to have an orphaned process, particularly if the app is set up as a headless console app or a windows service, where you don’t see anything on your screen to give you an indication of what’s running that has a hold on the files in your project’s /bin directory.

The Solution

The solution is a few things:

  1. Open Task Manager and close any orphaned processes you may have spun up -or- restart your machine to make sure everything is cleaned out.
  2. Ensure that all the projects are set to a debug configuration.  This is probably what you want, and will fix the could not copy error permanently.  If this is not what you want, then leave it, but now at least you’re aware of the cause.

I hope this helps point you in the right direction, particularly on projects where you may have a webapp and then a scheduler all in the same solution, but have a mix of debugger/no debugging for them and start to have the “Could not copy dll” error message.

 

 

Azure Functions: Developer Infrastructure is the Sweet Spot

I recently gave a talk at TriDev about Azure Functions, the serverless programming product from Microsoft. Azure Functions is basically functions as a service. You can basically write a single function in JavaScript or C# and it manages the entire infrastructure around it. You don’t need to worry about scaling, networking, load balancing, even containers. In my time evaluating it, one of the things I found was that the best and easiest application for easing into using Azure Functions is as part of your continuous delivery infrastructure.  A lot of times it’s scary to try new cloud services, or at minimum get experience with these new technologies in a “real” setting. Azure functions are perfect for getting developer utilities into production without having to manage our own server.

Here are the slides from my talk.

Building Great Software Teams

Over the last decade I’ve worked with dozens of teams helping them work better as a team and individuals through mentoring and helping build infrastructure.  I recently gave a presentation that used Maslow’s Hierarchy of Needs and compared that to how you can build great developers on your team. Take a look at the slides below and get in touch if you’d like me to give this talk at your local meetup or conference!

The Illusion of Code Quality

Code quality is something that I believe every developer strives for.  We want code to be the best it can be and there are tons of opinions on things developers can do to make quality high. Over the years as teams have moved to Agile from Waterfall, and as build and test automation has become better, a lot of the code quality metrics that experts have developed are becoming less helpful, or, dare I say, counter-productive.

The larger a team gets, but more importantly, the higher turnover gets (developers leaving the team/company and new developers without context come on to the team/are hired), the harder it is for code to remain high quality over time. We’re all human, we can’t keep everything in our head, we can’t mind-read the original developer who left the company and wrote this code. The worst part is that we don’t know what we don’t know. We duplicate effort because we didn’t know there was a design document on Box, or we don’t go update it.  There’s also setup information on the Wiki that should be changed, but we’ve not asked anyone where it is yet, because we didn’t know to, so our sweeping changes to the project aren’t reflected there.  We’re also on a deadline, and there were already existing comments that StyleCop saw, and it can’t automatically tell me that something in my code is now out of sync with the comments, so now developers can’t make any assumptions about the comments being right.  Any of this sound familiar?

That’s ok, though! That’s human nature! We aren’t computers (I’m glad we’re not) and we’re not good at keeping documentation in sync, especially when most teams now use Agile (I say that loosely), but still carry assumptions over from the Waterfall days that are literally duplicated with automation today.  So what should we do to help keep down the illusion of quality and actually introduce REAL quality into our code? (more…)

Azure Bot Service and Why You Should Check It Out

At Build last year (2018) I had some free time and dropped into a chatbot presentation. I’ll be honest, I didn’t really care much about chatbots. I use them a bit in slack, but honestly, they aren’t overly helpful. “/giphy nick cage” will find a gif for me, but it’s usually faster to Google things, particularly when I’m at a keyboard. Skype and facebook bots, in general, have always felt more like a choose your own adventure book than typing to an actual person and, again, there are usually faster UIs for finding information like that. So why is Microsoft’s service for building chatbot’s so compelling to me now? I’ll give it to you with one word: (more…)

Personal Branding for Software Developers

I recently gave a talk on branding for software developers at TriJS.  While I didn’t get a chance to record it, I did upload the slides for your viewing please.  The slides are readable without listening to the talk and have a few protips and specific actionable things that you can do to boost your personal brand as a developer.  It’s one of those things where you don’t really need it until you want to change jobs or you need to promote something, then you wish you did.  If you don’t know how to get started, take a look at the slides and let me know some of the things that you’ve done to increase your visibility as a software developer!

Automating the Web with Selenium and WebDriver

I recently gave a talk on browser automation and using Selenium with WebDriver.  Not only that, I gave a demo of using Selenium, WebDriver, xUnit, and creating a fluent API to create a framework that makes tests easy to compose.  Check them out and let me know what you think!  Selenium is a fantastic way to get a ton of value from automated front-end tests.