THE SQL Server Blog Spot on the Web

Welcome to SQLblog.com - The SQL Server blog spot on the web Sign in | |
in Search

Linchi Shea

Checking out SQL Server via empirical data points

Add-Content and Out-File are not for performance

When you write Powershell scripts and need to write a text file, you have a number of ways to accomplish that. The often suggested approach is to use cmdlets Add-Content or Out-File. Well, this is not news.

 

But some may not notice that these cmdlets are not there for performance. Sure, they are convenient. But if you need to write to a file many times, you may want to consider something else. To demonstrate the point, try the following script, which tests the performance of writing to a text file 10,000 times:

 

# Measure Add-Content

$t = get-date;

for ($a=1; $a -le 10000; $a++)

{

   add-content -path "c:\junk\test1.txt" -value "Hello";

}

$s = $(get-date).subtract($t).TotalMilliseconds;

write-output ("Add-Content:  " + $s);

 

# Measure Out-File

$t = get-date;

for ($a=1; $a -le 10000; $a++)

{

   "Hello" | out-file -filepath "c:\junk\test2.txt" -append

}

$s = $(get-date).subtract($t).TotalMilliseconds;

write-output ("Out-File:     " + $s);

 

 

# Measure StreamWriter

$t = get-date;

$sw = New-Object System.IO.StreamWriter "c:\junk\test3.txt";

for ($a=1; $a -le 10000; $a++)

{

    $sw.WriteLine("Hello")

}

$sw.Close();

$s = $(get-date).subtract($t).TotalMilliseconds;

write-output ("StreamWriter: " + $s);

 

The output from this script is similar to what I obtained from my laptop and shown below:

 

Add-Content:  34015.8427

Out-File:     37781.4918

StreamWriter: 781.255

 

From running this scripts multiple times on my laptop, using StreamWriter was consistently about 4 ~ 5 times faster than either Add-Content or Out-File. This should not really be surprising because each time you call Add-Content or Out-File, you open and close the file, and that is expensive.

 

Performance wise, using Add-Content or Out-File is similar to using StreamWriter in the following way (i.e. including opening and closing the file inside the loop):

 

# Measure StreamWriter, open/close the file for every write

$t = get-date;

for ($a=1; $a -le 10000; $a++)

{

    $sw = New-Object -typename System.IO.StreamWriter("c:\junk\test4.txt", "true");

    $sw.WriteLine("Hello")

    $sw.Close();

}

$s = $(get-date).subtract($t).TotalMilliseconds;

write-output ("StreamWriter2:" + $s);

 

And if you run the above script, you should find its performance very similar to that of Add-Content or that of Out-File.

 

So what is the takeaway? Well, do not use Add-Content or Out-File if performance matters such as when you write a massive number of lines to a file.

 

Published Monday, January 04, 2010 5:31 PM by Linchi Shea

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

 

Nick Beagley said:

That's very handy to know and quiet timely. Thanks for the simple example.

January 6, 2010 9:02 PM
 

Peter said:

That came in really, really handy for something I was just doing. I can't believe the time difference. Thanks for sharing.

March 16, 2010 7:43 PM
 

Lonely programmer said:

You are the man!

I could bring down the creation time of a file from 14 minutes (!) to less than 10 seconds.

Great!

February 14, 2013 4:09 AM
 

Tim said:

Thank you, this is exactly the information that I needed!

September 16, 2013 3:15 PM

Leave a Comment

(required) 
(required) 
Submit

About Linchi Shea

Checking out SQL Server via empirical data points

This Blog

Syndication

Powered by Community Server (Commercial Edition), by Telligent Systems
  Privacy Statement