This is an incredibly quick post so that I can remember what to do, not if, but when, I run into this again.
It seems that McAfee gets pretty excited when you try to run an npm install sometimes and will attempt to scan files during the process. This causes a lock on certain files, which if npm isn’t able to access them after so long, it will just give up. To remedy this, you’ll need to edit the npm config file. To do this easily, open npm’s config from the npm itself on the command line, like so:
npm config edit
Once the config file is open, look for this line:
; cache-lock-retries=10
Now, replace that line with the following, to significantly increase the cache lock retries:
cache-lock-retries=1000
This will allow npm to keep retrying past the point in which McAfee (or other antivirus software) has released its own lock. The only real potential downside here is that if something truly has a permanent lock that it can’t recover from, the npm install won’t fail until much longer, though I’d take “running slow because I have antivirus” over “not able to do an npm install because of antivirus” any day.
Hope this helps and let me know if you know of another workaround that’s faster! Thanks for stopping by!
Recently, I ported this website to Gatsby.js. As part of it, I wanted to make sure that I tied up any loose ends and cleaned up all of my accessibility and technical issues. One great way to do that is to run a tool called Webhint over your website and get "hints" as to what you need to change. These hints are installed through NPM, and if you’re familiar with NPM, you probably know the node_packages folder ends up with hundreds of package dependencies. One of these dependencies ended up needing node-gyp, which is a build tool that requires a few build tools (like VC++ and python) installed on your machine, particularly if you’re on Windows. This article will explain the how to fix the error both quickly and permanently.
Here’s what I ran from the command line and the error message that I received. **You’ll see that it’s attempting to run node-gyp and then gives an error that it can’t find VBuild.exe.
RyannosaurusRex@Bespin MINGW64 /c/git/personal/ryanhayes.net (master)
$ npm install
> [email protected] install C:\git\ryanhayes.net\node_modules\iltorb
> node ./scripts/install.js || node-gyp rebuild
info looking for cached prebuild @ C:\Users\RyannosaurusRex\AppData\Roaming\npm-cache\_prebuilds\ff68b8-iltorb-v2.4.2-node-v67-win32-x64.tar.gz
http request GET https://github.com/MayhemYDG/iltorb/releases/download/v2.4.2/iltorb-v2.4.2-node-v67-win32-x64.tar.gz
http 404 https://github.com/MayhemYDG/iltorb/releases/download/v2.4.2/iltorb-v2.4.2-node-v67-win32-x64.tar.gz
WARN install No prebuilt binaries found (target=11.11.0 runtime=node arch=x64 libc= platform=win32)
C:\git\ryanhayes.net\node_modules\iltorb>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Warning: unrecognized setting VCCLCompilerTool/MultiProcessorCompilation
Warning: unrecognized setting VCCLCompilerTool/MultiProcessorCompilation
Warning: unrecognized setting VCCLCompilerTool/MultiProcessorCompilation
Warning: unrecognized setting VCCLCompilerTool/MultiProcessorCompilation
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
MSBUILD : error MSB3428: Could not load the Visual C++ component "VCBuild.exe". To fix this, 1) install the .NET Framework 2.0 SDK, 2) install
Microsoft Visual Studio 2005 or 3) add the location of the component to the system path if it is installed elsewhere. [C:\git\ryanhay
es.net\node_modules\iltorb\build\binding.sln]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:262:23)
gyp ERR! stack at ChildProcess.emit (events.js:197:13)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:254:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js"
"rebuild"
gyp ERR! cwd C:\git\ryanhayes.net\node_modules\iltorb
gyp ERR! node -v v11.11.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
npm WARN [email protected] requires a peer of styled-components@>= 2 but none is installed. You must install peer dependencies yourself.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node ./scripts/install.js || node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\RyannosaurusRex\AppData\Roaming\npm-cache\_logs\2019-04-22T03_31_15_972Z-debug.log
The ‘Could not load the Visual C++ component "VCBuild.exe"’ solution
The error message mentions that you need to install MSBuild or Visual Studio. That works, of course, and is one solution. If you don’t need all of Visual Studio, that’s a very large footprint (7+ GB of storage) just to satisfy a small dependency like VCBuild.exe.
If you already have Visual Studio or the necessary tools installed, make sure you are using powershell. To swith to powershell on the linux/WSL command line, just run $ powershell to switch and try again.
Installing Windows Build Tools
To fix the issue without having to install Visual Studio, Microsoft has created a great npm package called windows-build-tools. This package allows you to compile Native Node modules, which is required by some npm packages, particularly packages that deal with Chromium and canvas.
Here’s the output from the run on my machine. You’ll see that it installs VCBuild.exe and related tools, but also Python as well, which is necessary for a lot of other npm packages.
PS C:\windows\system32> npm install --global windows-build-tools
> [email protected] postinstall C:\Users\RyannosaurusRex\AppData\Roaming\npm\node_modules\windows-build-tools
> node ./dist/index.js
Downloading python-2.7.15.amd64.msi
[> ] 0.0% (0 B/s)
Downloaded python-2.7.15.amd64.msi. Saved to C:\Users\RyannosaurusRex\.windows-build-tools\python-2.7.15.amd64.msi.
Downloading vs_BuildTools.exe
[> ] 0.0% (0 B/s)
Downloaded vs_BuildTools.exe. Saved to C:\Users\RyannosaurusRex\.windows-build-tools\vs_BuildTools.exe.
Starting installation...
Launched installers, now waiting for them to finish.
This will likely take some time - please be patient!
Status from the installers:
---------- Visual Studio Build Tools ----------
2019-04-21T23:32:57 : Verbose : [InstalledProductsProviderImpl]: Stream was closed
2019-04-21T23:32:57 : Verbose : [InstallerImpl]: Rpc connection was closed.
2019-04-21T23:32:57 : Verbose : [InstallerImpl]: Stream was closed
2019-04-21T23:32:57 : Verbose : Completed notifying all clients that the update is starting
2019-04-21T23:32:57 : Verbose : Updating installer [bootstrapperArguments: /finalizeInstall install --in C:\ProgramData\Microsoft\VisualStudio\Packages\_bootstrapper\vs_setup_bootstrapper_201904212332293329.json --norestart --quiet --includeRecommended --add Microsoft.VisualStudio.Workload.VCTools --locale en-US --activityId 56083b59-412f-473c-bdae-1976d5623e0d]
------------------- Python --------------------
Successfully installed Python 2.7
Wrapping up
While the error message is fairly clear, the solution it proposed was not a great one. Visual Studio is a very large dependency to take, particularly for a lightweight Gatsby site. Thankfully, windows-build-tools allowed me to install just the dependencies that I needed without the extra bloat.
I run the software systems Harvest Community Church and also play guitar for our 3 services on Sunday. We’ve been using Planning Center’s Services app to schedule and manage our Sunday mornings for almost a decade. Recently Harvest moved from Rock RMS to PCO, and it’s been great. Below is the database script I used to extract the data needed from RockRMS and get it in the format needed for a Planning Center Online import.
Extract the data from Rock RMS
Keep in mind that this script has a “SELECT TOP(1000)” at the top because we had A LOT of data. It’s easier to test your export if you don’t pull out everyone every time. When ready, you’ll want to replace “SELECT TOP(1000)” WITH “*” in order to pull all of the records. You’ll notice that some of the lines are commented out for the migration from Rock RMS to PCO. These records we ended up not needing or were unused. If you do need them, those names are what PCO is looking for, so just add to the script to pull those additional pieces if needed.
Once you have the data extracted, you’ll need to get it into a CSV format. I like to use Azure Data Studio (Download here) to connect to the database and run the query because you can save the results of the above query as a CSV, which is the format Planning Center needs in order to do an import.
Importing into Planning Center Online
Once you have the data in a .csv file, log into your Planning Center account, go to the People app, and select CSV Import / Export and upload your file.
Once the file is uploaded, you can review the import. You can revert it within the first few hours in case things don’t look quite.
Cleanup and Gotchas
From there, you may need to merge some people if you already have folks in the system. Keep an eye out for any Services people that you merge and make sure the email they used in Services remains their primary email. If it doesn’t, they may stop getting notifications for when they’re scheduled to serve or be unable to log in.
Hope this helps your Rock RMS to Planning Center Online migration go a little more smoothly!
McAfee Endpoint Security and I have a love/hate relationship in that I hate it when it gets in my way and love it when it’s not installed. In general, I appreciate security and security research, but recently I had been trying out (or attempting to try out) Docker and Kubernetes for a project I’m working on. It’s a .NET 4.6 web application, and as such, requires Windows Server Core (as opposed to the much lighter-weight, new Windows Nano Server or a Linux-based container). The fact that we now have an option to effectively containerize any application, including a windows application, is incredible. So yesterday, I decided to try and set it up. Here’s how it went.
The only potential downside that I had heard about Windows Server Core containers were that they were pretty big. When I say big I mean like real big. 10GB big.
Any issues with the size?…we were playing with those and they were up into the 10 gig range.
Ok, fine, that’s…that’s not good but there’s no other way to run full framework .NET apps without it a WindowsServerCore container, so I can get past that.
Installing Docker and gotchas
Installing Docker on Windows with McAfee is pretty straightforward. This was the first time I’d installed it in a couple years, so I was pleasantly surprised when the entire thing was pretty seamless. The wizard created the necessary docker-users security group and added me to it and has a pretty nice interface now. There were a few initial setup steps that were unclear I had to lookup, though:
You need to add your user to the Hyper-V Administrators group. For whatever reason, the Docker for Windows installation created the docker-users group and added me there, but didn’t to the existing Hyper-V Administrators group. 🤷
When building a docker container via the command line interface (Docker CLI), the login information is different from what you use to log into the website. The website accepts you username OR your email address (I use my email address), but the CLI will only accept your username, and not the email address. It also doesn’t tell you why the login fails, so just get in a habit of using your username.
Issues building a docker image with McAfee Endpoint Security installed
Everything seemed to be going as well as I could expect until it came time to run docker build on my dockerfile and get everything set up. It seemed to be pulling things from the registry OK, but then I was met with this error message:
C:\WINDOWS\system32> docker pull microsoft/windowsservercore
Using default tag: latest
latest: Pulling from microsoft/windowsservercore
9c7f9c7d9bc2: Extracting [==================================================>] 3.738 GB/3.738 GB
d33fff6043a1: Download complete
failed to register layer: rename C:\ProgramData\Docker\image\windowsfilter\layerdb\tmp\write-set-925881297 C:\ProgramData\Docker\image\windowsfilter\layerdb\sha256\3fd27ecef6a323f5ea7f3fde1f7b87a2dbfb1afa797f88fd7d20e8dbdc856f67: Access is
denied.
Ok, so let’s look at what’s happening here. It’s saying it can’t register a layer because access is denied on my computer’s file system…but I’m administrator! What could possibly be the issue? Then I remember. McAfee. A quick Google search showed a lot of other folks having the same kind of issue and ends up that McAfee doesn’t even support Windows containers. Yea, that’s right.
McAfee Endpoint Security Does Not Support Windows-based Docker Containers!
After tons of research, the answer from McAfee itself (according to a knowledge base article), is that McAfee Endpoint Security does not support Windows Docker containers! It seems the main issues were:
How can I run Windows Docker Containers on Windows if McAfee doesn’t support it?
There are actually a few workarounds that have been successful for me and my team:
Virtual Machines
First, you can use the docker tools inside a separate “real” virtual machine (using VMWare, VirtualBox, etc) and actually perform the work there. This gets you around McAfee being on the VM directly, but also has a few drawbacks. It will slow your workflow down as you’ll have the VM to jump through from your local machine to the docker tools and images you may be running. This is not ideal, but might give you some runway until McAfee can add support.
.NET Core!
McAfee says that Linux-based containers are OK. This means if you’re using a language or tools that require Windows containers, you may be able to move to something that doesn’t require them.
I know this isn’t much help, particularly if you have a large application already that you’re moving to Docker, but can be an option if you’re just starting out or if you are able to adjust your stack enough to be able to drop the Windows requirement.
Remove McAfee?
Move away from McAfee if Windows Docker containers are a hard requirement for you and no other options work well for your team.
The bottom line
Basically, McAfee is really tough to work around as a developer, and Windows Docker containers are no exception. You have a limited options in the meantime, but I feel like the best recommendation is to not waste your time with workarounds and wait it out. McAfee should have an update at some point in the future to fix the issue. If you can’t afford to wait and the workarounds aren’t options, remember that making your voice heard on Twitter and Email and letting McAfee know this is important will go a long way in having them increase this as a priority.
Recently I had an issue where when I would try to run and debug a web app inside Visual Studio 2017, I would get an error where the .dlls couldn’t be copied out to the bin directory, which meant I couldn’t actually run the app at all! I would keep getting this error:
“Could not copy “C:\MyProject\MyDLL.dll” to “C:\MyProject\bin\Debug\MyDLL.dll”
This error is usually caused by a lock on the target file, preventing it from being deleted.
There was only one problem – I had restarted Visual Studio, and it was still happening.
I even restarted my computer, which fixed the problem once, but subsequent runs brought the error back.
The Root Cause
The actual root cause ended up that I actually had multiple projects set up as my startup project, but ONE of the projects wasn’t set up to debug, while the other was.
When you set the configuration in Visual Studio for a project to run without debugging, it actually doesn’t give the app the hooks that Visual Studio needs to close the app once you select “Stop Debugging.” This causes you to have an orphaned process, particularly if the app is set up as a headless console app or a windows service, where you don’t see anything on your screen to give you an indication of what’s running that has a hold on the files in your project’s /bin directory.
The Solution
The solution is a few things:
Open Task Manager and close any orphaned processes you may have spun up -or- restart your machine to ensure everything is cleaned out.
Ensure that all the projects are set to a debug configuration. This is probably what you want, and it will fix the could not copy error permanently. If this is not what you want, leave it, but now you’re aware of the cause.
I hope this helps point you in the right direction, particularly on projects where you may have a web app and then a scheduler all in the same solution, but have a mix of debugger/no debugging for them and start to have the “Could not copy dll” error message.
I recently gave a talk at TriDev about Azure Functions, the serverless programming product from Microsoft. Azure Functions is basically functions as a service. You can basically write a single function in JavaScript or C# and it manages the entire infrastructure around it. You don’t need to worry about scaling, networking, load balancing, even containers. In my time evaluating it, one of the things I found was that the best and easiest application for easing into using Azure Functions is as part of your continuous delivery infrastructure. A lot of times it’s scary to try new cloud services, or at minimum get experience with these new technologies in a “real” setting. Azure functions are perfect for getting developer utilities into production without having to manage our own server.
Over the last decade I’ve worked with dozens of teams helping them work better as a team and individuals through mentoring and helping build infrastructure. I recently gave a presentation that used Maslow’s Hierarchy of Needs and compared that to how you can build great developers on your team. Take a look at the slides below and get in touch if you’d like me to give this talk at your local meetup or conference!
Code quality is something that I believe every developer strives for. We want code to be the best it can be and there are tons of opinions on things developers can do to make quality high. Over the years as teams have moved to Agile from Waterfall, and as build and test automation has become better, a lot of the code quality metrics that experts have developed are becoming less helpful, or, dare I say, counter-productive.
The larger a team gets, but more importantly, the higher turnover gets (developers leaving the team/company and new developers without context come on to the team/are hired), the harder it is for code to remain high quality over time. We’re all human, we can’t keep everything in our head, we can’t mind-read the original developer who left the company and wrote this code. The worst part is that we don’t know what we don’t know. We duplicate effort because we didn’t know there was a design document on Box, or we don’t go update it.  There’s also setup information on the Wiki that should be changed, but we’ve not asked anyone where it is yet, because we didn’t know to, so our sweeping changes to the project aren’t reflected there.  We’re also on a deadline, and there were already existing comments that StyleCop saw, and it can’t automatically tell me that something in my code is now out of sync with the comments, so now developers can’t make any assumptions about the comments being right.  Any of this sound familiar?
That’s ok, though! That’s human nature! We aren’t computers (I’m glad we’re not) and we’re not good at keeping documentation in sync, especially when most teams now use Agile (I say that loosely), but still carry assumptions over from the Waterfall days that are literally duplicated with automation today. Â So what should we do to help keep down the illusion of quality and actually introduce REAL quality into our code?
At Build last year (2018) I had some free time and dropped into a chatbot presentation. I’ll be honest, I didn’t really care much about chatbots. I use them a bit in slack, but honestly, they aren’t overly helpful. “/giphy nick cage” will find a gif for me, but it’s usually faster to Google things, particularly when I’m at a keyboard. Skype and facebook bots, in general, have always felt more like a choose your own adventure book than typing to an actual person and, again, there are usually faster UIs for finding information like that. So why is Microsoft’s service for building chatbot’s so compelling to me now? I’ll give it to you with one word:
