• Download Dokan – 519.76 KB
• Download Dokan – 532.52 KB

Introduction

Hiroki Asakawa created a device driver that enables an application in user-mode to simulate a file-system,
and distributed it under a MIT-style license. This post shows how that’s used in C# to create
the functionality of a RAM-Disk, with some tips on rolling your own file-system application.

Before you can run the code, you have to run the installer to install a proxy driver (dokan.sys). This driver then acts
as an intermediate between the kernel and our custom .NET solution. The installer for the proxy driver is also included with the
sourcecode here, but I’d recommend downloading the package from the original website as they contain two cool examples;
The first implements a mirror of your C-drive, and the second makes the Registry available as a readonly
drive. This article briefly explains how the Dokan-libraries are used to build something that resembles a
RAM-disk in functionality

Having the option to expose data as a file-system has several advantages. It
makes your data instantaneous available for other applications, without
the need for a complicated UI.

Download the basic installer and .NET bindings to Dokan
here[^].

Using the code

Log in as administrator, run the installer (it’s just 513 Kb), open the project and hit F5; a trayicon should appear, and on
doubleclick it should open the Windows-Explorer, pointing to a simulated harddisk. It’ll mount on
the first available free drive-letter. Paste a zipfile on there and extract it

I choose to implement a basic RAM-Disk to test the library (version 0.5.3), with the results documented here.

There’s three projects in the solution;

• DokanNet – these are the .NET bindings to the Dokan libraries
• Buffers – used to replace the MemoryStream
• Dokan.Mem – example-implementation of the Dokan-interface, simulating a RAM-Disk

DokanNet

This project contains the bindings to the Dokan-libraries, provided by the
DokanOperations-interface. It’s a pretty straightforward definition of
the actions that an application (like Word) can do on the file-system.

  1  public interface DokanOperations
  2  {
  3      int CreateFile(string filename, ..., DokanFileInfo info);
  4      int OpenDirectory(string filename, DokanFileInfo info);
  5      int CreateDirectory(string filename, DokanFileInfo info);
  6      int Cleanup(string filename, DokanFileInfo info);
  7      int CloseFile(string filename, DokanFileInfo info);
  8      int ReadFile(string filename, ..., DokanFileInfo info);
  9      int WriteFile(string filename, ..., DokanFileInfo info);
 10      int FlushFileBuffers(string filename, DokanFileInfo info);
 11      int GetFileInformation(string filename, ..., DokanFileInfo info);
 12      int FindFiles(string filename, ArrayList files, DokanFileInfo info);
 13      int SetFileAttributes(string filename, ..., DokanFileInfo info);
 14      int SetFileTime(string filename, ..., DokanFileInfo info);
 15      int DeleteFile(string filename, DokanFileInfo info);
 16      int DeleteDirectory(string filename, DokanFileInfo info);
 17      int MoveFile(string filename, ..., DokanFileInfo info);
 18      int SetEndOfFile(string filename, long length, DokanFileInfo info);
 19      int SetAllocationSize(string filename, long length, DokanFileInfo info);
 20      int LockFile( string filename, long offset, long length, DokanFileInfo info);
 21      int UnlockFile(string filename, long offset, long length, DokanFileInfo info);
 22      int GetDiskFreeSpace(ref ulong freeBytesAvailable, ..., DokanFileInfo info);
 23      int Unmount(DokanFileInfo info);
 24  }

Once you have a class based on this interface (e.g. ‘MyDokanOperations‘),
you can launch your new drive by calling the Dokan-main method. The drive will be available
as long as this (blocking) task is running.

A skeleton-application is given below, implementing the DokanOperations
in the MyDokanOperations class. Take note that the DokanOperations is
an interface! It might not be named IDokanOperations, but that’s what it should read.

  1  class MyDokanOperations : DokanOperations
  2  {
  3     // implementation of DokanOperations interface goes here..
  4  }
  5  
  6  DokanOptions options = new DokanOptions
  7  {
  8  	DriveLetter = 'Z',
  9  	DebugMode = true,
 10  	UseStdErr = true,
 11  	NetworkDrive = false,
 12  	Removable = true,     // provides an "eject"-menu to unmount
 13  	UseKeepAlive = true,  // auto-unmount
 14  	ThreadCount = 0,      // 0 for default, 1 for debugging
 15  	VolumeLabel = "MyDokanDrive"
 16  };
 17  
 18  static void Main(string[] args)
 19  {
 20  	DokanNet.DokanMain(
 21  		options,
 22  		new MyDokanOperations());
 23  }

The examples all show a console, which is cool when you’re developing your
new file-system. It’s very useful when debugging, and you can actually follow
the interaction between the kernel and the simulated file-system.

Dokan.Mem

The RAM-Disk also shows the console-window, but you easily disable it by
changing the output-type of the project to “Windows Application”. I’ve provided a
trayicon that let’s one interact with the application in release-mode.

Back to that DokanOperations-interface; Most of these calls have
actual API counterparts and you can find a description on MSDN. The API-description
is very useful as it documents the general flow, and lists the error-codes that it might return.

Let’s take a look at the actual implementation of the DeleteFile
[^]
method;

  1  public int DeleteFile(string filename, DokanFileInfo info)
  2  {
  3  	// get parent folder
  4  	MemoryFolder parentFolder = _root.GetFolderByPath(
  5  		filename.GetPathPart());
  6  
  7  	// exists?
  8  	if (!parentFolder.Exists())
  9  		return -DokanNet.ERROR_PATH_NOT_FOUND;
 10  
 11  	// fetch file;
 12  	MemoryFile file = parentFolder.FetchFile(
 13  		filename.GetFilenamePart());
 14  
 15  	// exists?
 16  	if (!file.Exists())
 17  		return -DokanNet.ERROR_FILE_NOT_FOUND;
 18  
 19  	// delete it;
 20  	parentFolder.Children.Remove(file);
 21  	file.Content.Dispose();
 22  
 23  	return DokanNet.DOKAN_SUCCESS;
 24  }

MSDN states for the DeleteFile function;

“If an application attempts to delete a file that does not exist, the DeleteFile function fails with ERROR_FILE_NOT_FOUND. If the file is a read-only file, the function fails with ERROR_ACCESS_DENIED.”

The RAM-disk does check whether the file exists, but I didn’t implement the check on the
file-attributes yet. As far as setting the file-attributes go, that’s not implemented at all;

  1  public int SetFileAttributes(
  2  	string filename,
  3  	FileAttributes attr,
  4  	DokanFileInfo info)
  5  {
  6  	return -DokanNet.DOKAN_ERROR;
  7  }

If you take a look at the interface again, you’ll notice that there’s no OpenFile method.
If the system wants to open a file, it will call the CreateFile method. It takes the same
flags that the
CreateFile
[^] API uses.
Depending on the FileMode, we create or open an existing or a non-existing file;

  1  public int CreateFile(
  2      string filename,
  3      FileAccess access,
  4      FileShare share,
  5      FileMode mode,
  6      FileOptions options,
  7      DokanFileInfo info)
  8  {
  9      [...]
 10  
 11       // attempt to use the file
 12      switch (mode)
 13      {
 14          // Opens the file if it exists and seeks to the end of the file,
 15          // or creates a new file
 16          case FileMode.Append:
 17              if (!thisFile.Exists())
 18                  MemoryFile.New(parentFolder, newName);
 19              return DokanNet.DOKAN_SUCCESS;
 20  
 21          // Specifies that the operating system should create a new file.
 22          // If the file already exists, it will be overwritten.
 23          case FileMode.Create:
 24              //if (!thisFile.Exists())
 25                  MemoryFile.New(parentFolder, newName);
 26              //else
 27              //	thisFile.Content = new Thought.Research.AweBuffer(1024); //MemoryStream();
 28              return DokanNet.DOKAN_SUCCESS;
 29  
 30          // Specifies that the operating system should create a new file.
 31          // If the file already exists, an IOException is thrown.
 32          case FileMode.CreateNew:
 33              if (thisFile.Exists())
 34                  return -DokanNet.ERROR_ALREADY_EXISTS;
 35              MemoryFile.New(parentFolder, newName);
 36              return DokanNet.DOKAN_SUCCESS;
 37  
 38          // Specifies that the operating system should open an existing file.
 39          // A System.IO.FileNotFoundException is thrown if the file does not exist.
 40          case FileMode.Open:
 41              if (!thisFile.Exists())
 42                  return -DokanNet.ERROR_FILE_NOT_FOUND;
 43              else
 44                  return DokanNet.DOKAN_SUCCESS;
 45  
 46          // Specifies that the operating system should open a file if it exists;
 47          // otherwise, a new file should be created.
 48          case FileMode.OpenOrCreate:
 49              if (!thisFile.Exists())
 50                  MemoryFile.New(parentFolder, newName);
 51              return DokanNet.DOKAN_SUCCESS;
 52  
 53          // Specifies that the operating system should open an existing file.
 54          // Once opened, the file should be truncated so that its size is zero bytes
 55          case FileMode.Truncate:
 56                  if (!thisFile.Exists())
 57                      thisFile = MemoryFile.New(parentFolder, newName);
 58                  thisFile.Size = 0;
 59                  return DokanNet.DOKAN_SUCCESS;
 60      }
 61  
 62      return DokanNet.DOKAN_ERROR;
 63  }

The MemoryFolder and MemoryFile classes are used to
map the “files” in memory in a hierarchical structure. There’s a rootnode that
represents the root of the drive, and may contain objects that represents either a file or a folder;

You’ll notice that the MemoryFile class is abstract.

Buffers

The RAM-Disk prototype was originally based on a MemoryStream. That would
make it a “virtual memory disk”, we’re missing some functionality before we can call it a RAM-Disk.

I’ve
replaced the MemoryStream with a buffer that’s based on an idea
[^]
from David Pinch. He wrote a class
that can be used to allocate memory, freeing it in the Dispose section.
It’s originally designed to provide the ability to protect an allocated region, hence
the name ProtectedBuffer. This block of memory can be accessed as a stream,
with the drawback that it can’t be resized.

The AweBuffer class is based on that ProtectedBuffer-class,
adding a call to the
VirtualLock
[^]
API. This way the information is pinned into physical-memory, for as long as the
thread is running*. Without this, the RAM-disks’ maximum size would only be restricted by
the amount of available virtual memory. In theory, it would speed up the access to the data.

In practice, the RAM-disk tends to push all other running applications into
the swapfile, creating an even larger delay. It would require some testing on
different machines to get a decent indication, but I think that this buffer
performs worse than a simple MemoryStream.

*) See the VirtualLock entry on The Old New Thing
[^]

Whether the system uses a MemoryStreamFile or a AweMemoryFile is
determined when the application starts. I figured that there would be more people
who’d want to try the difference, so it’s easy to switch between RAM-Disk mode (using the AweBuffer)
and Virtual Disk mode (using the MemoryStream).
Simply toggle the bool in the Main-method of the Dokan.Mem project and recompile;

  1  [STAThread()]
  2  static void Main(string[] args)
  3  {
  4      // Set us up a tray-icon to interact with the user;
  5      SetupNotifyIcon();
  6  
  7      // Set to false if you want to test with the AweBuffers
  8      // instead of a MemoryStream
  9      MemoryFile.UseMemStream = true;

Points of Interest

In general

• You need to be logged in as an Administrator to run the file-system

• The code presented is of prototype-quality; there’s no locking, no errorhandling, and just a very rudimentary UI.

• Paths and filenames are case-insensitive in Windows; different apps will use different casings when
  asking for your files.

• There’s no special method to rename files (or folders), that’s done by the
  MoveFile method.

• The folders are usually ordered in a hierarchy, but that’s not required. One could omit
  the folderstructure completely and only show files (see the FindFiles method).

• You might want to disable any viruscheckers, as they tend to scan each file. The first file
  being requested after mounting is “AutoRun.Inf”

• The MemoryFile classes have the attribute
  FILE_ATTRIBUTE_NOT_CONTENT_INDEXED
  [^] to
  prevent Windows from content-indexing the files.

• The online
  readme
  [^]
  contains a more in-depth explanation on the inner structure of the Dokan-libraries and
  how they work.

If you are interfacing with Sql Server

Because it’s cool to be able to use Paint.NET over the explorer on a picture that’s
stored in Sql Server.

There’s Stefan Delmarco’s
VarBinaryStream[^],
providing convenient stream-based access to a VarBinary field.
An implementation would go along these lines;

  1  public int ReadFile(string sourcePath, byte[] buffer, ref uint readBytes,
  2      long offset, DokanFileInfo info)
  3  {
  4      using (var con = new SqlConnection())
  5      {
  6          int fileId = GetFileIdByPath(sourcePath);
  7          using (var myVarBinarySource = new VarBinarySource(
  8              con,
  9              "[TableName]", // source table
 10              "[Contents]",  // source column
 11              "[Id]",        // key column
 12              resourceId))
 13          {
 14              var dbStream = new VarBinaryStream(myVarBinarySource);
 15              dbStream.Seek(offset, SeekOrigin.Begin);
 16              readBytes = (uint) dbStream.Read(buffer, 0, buffer.Length);
 17          }
 18      }
 19      return DokanNet.DOKAN_SUCCESS;
 20  }

That’s opening and closing databaseconnections like crazy, but all in all, it performs quite
well. Alternatively, you could open the connection in the CreateFile method,
add it to a list, and close it again when the CloseFile method is called.
If you need testdata; the AdventureWorks database contains a table called [Production].[Document],
containing a few Word-documents as binary blobs.

Conclusion

Hiroki did a great job, the Dokan-libraries perform great. You can put breakpoints
all over the place, and step trough your code without any special settings

As for the RAM-Disk goes, it doesn’t add much speed. The benchmark crashed
when trying with the AweBuffer version, and it reported a mere 7 Mb/s when creating files.
The maximum-speed during read-operations was around 40 Mb/s. As a comparison, my harddisk
does around 64 Mb/s when creating files, with an average reading speed of 165 Mb/s.

History

• Initial version, 20-7-2010