The Windows NTFS file system is not a strong performer with small files in comparison with ext4. This is a large reason why operations like building a virtual environment in Python or installing a package take noticeably longer in Windows.

One way around this is to use WSL2, which is a different variant of WSL that runs under a specialized, lightweight VM. It provides a far more performant experience than WSL1 and alleviates other shortcomings of WSL1. See WSL2 Disk Performance for a performance comparison between the two.

Measuring performance

Pull up a WSL terminal and enter the following commands. Expect them to take a minute or two to execute and take note of the times.

mkdir test
cd test
dd if=/dev/zero of=testfile bs=1000 count=500
time split -b 10 -a 10 testfile
time rm -rf *

The above command generates 50k files by generating a 500KB file, splitting it into files of 10 bytes, and lastly removing all the files.

Disable 8.3 Naming

The 8.3 file naming convention was used in DOS and early versions of Windows. To maintain backward compatibility with programs over two decades old, Windows enumerates longer file names with the 8.3 conventions. This results in additional overhead that adds up when handling a large number of files.

Open the Windows command shell as Administrator. Enter the following to see if 8dot3 naming is enabled for your system drive.

PS C:\WINDOWS\system32> fsutil.exe 8dot3name query C:
The volume state is: 0 (8dot3 name creation is enabled).
The registry state is: 2 (Per volume setting - the default).

Based on the above settings, 8dot3 name creation is enabled on C:

The enumeration can be turned off at any time and may be turned off either per NTFS volume or for all volumes.

Here is how to disable the enumeration on all NTFS volumes:

PS C:\WINDOWS\system32> fsutil.exe behavior set disable8dot3 1
The registry state is now: 1 (Disable 8dot3 name creation on all volumes).

Disabling 8dot3 naming prevents 8.3 enumerations on files created after the change. Existing files in the directory created beforehand will continue to slow performance. Either re-create those files or check out Disable 8.3 Naming (and strip those short names too) on how to remove short names from the files.

Update real-time protection to exclude Linux processes

By default, Windows Defender will scan files being used by a process. This impacts processes that perform a lot of file operations. Some suggest to turn off real-time protection; a better way is to exclude files those processes are working with from being scanned in real-time.

Issue the which command in WSL to determine where the executable location of the process lives.

phil@small-kine:/mnt/c/dev/dd_test$ which split
/usr/bin/split
phil@small-kine:/mnt/c/dev/dd_test$ which rm
/bin/rm

Next, update the real-time protection to exclude that process or all processes in that folder. Shown below is how to exclude processes located in the /usr/bin and /bin directories.

The following assumes that there is only one distribution installed for WSL, Ubuntu. It is possible to have multiple distributions (Alpine, Debian, etc.) and multiple versions of them. For those situations, change the Where-Object query in the first command below to pick the correct WSL package.

Open Windows PowerShell as Administrator and enter the following commands.

PS C:\WINDOWS\system32> $wsl_target = [IO.Path]::Combine($env:LOCALAPPDATA, "Packages", (Get-AppxPackage | Where-Object -Property "Name" -Like "*Ubuntu*").PackageFamilyName, "LocalState", "rootfs")
PS C:\WINDOWS\system32> Test-Path $wsl_target
True
PS C:\WINDOWS\system32> Add-MpPreference -ExclusionProcess (Join-Path $wsl_target -ChildPath "/usr/bin/*")
PS C:\WINDOWS\system32> Add-MpPreference -ExclusionProcess (Join-Path $wsl_target -ChildPath "/bin/*")

Here is an example for excluding Node.js processes under NVM.

PS C:\WINDOWS\system32> Add-MpPreference -ExclusionProcess (Join-Path $wsl_target -ChildPath "/home/your_home_dir/.nvm/versions/node/*")

While WSL processes are run from directories under %localappdata%\Packages\..., they are not meant to be used for file access, use \\wsl$\ instead.

Check out Defender Anti-Virus Configure Process Exclusions for further configuration options for Windows Defender.

Performance

Using the file I/O example provided earlier and running a benchmark you can see a marked improvement in performance using the proceeding techniques. That said, there still is dramatic difference between WSL and Linux in raw performance.

Operation	WSL	WSL with changes	Docker (VM)
split 50k files	2m3.72s	39.95s	4.95s
rm 50k files	27.15s	18.71s	2.04s

Taking a look at some workflows and how their performance varies between systems those differences are not as substantial as before, but still noticeable.

Operation	WSL	WSL with changes	Docker (VM)
Python 3.6.9 venv creation	12.63s	5.86s	3.17s
npm create next-app	2m33.90s	1m13.31s	24.95s
next.js project build	45.91s	21.01s	20.83s

Benchmarks performed on a 1st generation Surface Pro with 64GB SSD. Your results will see a pronounced improvement with better hardware.

Node.js 12.13 LTS used for next.js benchmarks.