Using Parallel and ThreadPool / QueueUserWorkItem in Combination with Writing / Locking Files

If you’ve wanted to combine parallel / multi-threading and also writing to files, you’ve probably experienced the issue of a file being locked when another process is writing at the same time. Something like:

An unhandled exception of type ‘System.IO.IOException’ occurred in mscorlib.dll

Additional information: The process cannot access the file ‘c:\yourfile’ because it is being used by another process.

One of the ways to get around this is utilize the “lock” statement.

In my below example, I’m running parallel tasks (including the optional MaxDegreeOfParallelism) and writing to a file. I also added a stopwatch and a .sleep to add time and debugging to the method. If you were to remove the lock() from the below, you should experience the ioexception.

Hope this helps as you create parallel tasks (so many great uses) and write files (maybe some simple logging, etc.)

You can also download the sample here.

using System;
using System.Diagnostics;
using System.IO;
using System.Threading;
using System.Threading.Tasks;

namespace parallelTasks_Example
{
    //custom object for passing to queue task
    public class UpdateFile
    {
        public string lineData { get; set; }

        public string filePath { get; set; }

        public long timeTaken { get; set; }
    }

    internal class Program
    {
        //set an update to use a lock
        private static object fileLock = new object();

        private static void Main(string[] args)
        {
            //just for testing
            Stopwatch timer = new Stopwatch();

            timer.Reset();

            //get some data
            string[] contacts = { "jon", "jon2", "aaron", "efren", "diedrich", "tina", "sandy", "haylie", "trevor", "shondrella" };

            //do something with the data
            timer.Start();
            Parallel.ForEach(contacts, new ParallelOptions { MaxDegreeOfParallelism = 3 }, contact =>
                {
                    //do the task
                    Console.WriteLine(contact + " " + timer.ElapsedMilliseconds);
                    //create an object to past to the queue item
                    UpdateFile fileupdate = new UpdateFile { filePath = @"c:\temp\temp.txt", lineData = contact, timeTaken = timer.ElapsedMilliseconds };
                    //add the queue item
                    ThreadPool.QueueUserWorkItem(UpdateFile, fileupdate);
                });

            Console.ReadLine();
        }

        private static void UpdateFile(object fileData)
        {
            //set object to be locked
            lock (fileLock)
            {
                //get the file
                using (var fw = File.AppendText(((UpdateFile)fileData).filePath))
                {
                    //write the data
                    fw.WriteLine(((UpdateFile)fileData).lineData + ": " + ((UpdateFile)fileData).timeTaken.ToString());
                    //make process take long (for example)
                    System.Threading.Thread.Sleep(100);
                }
            }
        }
    }
}
Using Parallel and ThreadPool / QueueUserWorkItem in Combination with Writing / Locking Files