I have the following method in my console application which takes a list of files in SortedList (VMsPerDisk) and then copies them from their source to a remote share.
The files are usually fairly big so the entire copy can take several hours to complete. I'd like to implement multi-threading so that the CopyAFile method is launched on a separate thread for each file.
What is the best approach here and how to do I ensure the all the threads (i.e. copies) are complete? I guess I have the following options
Oh and I'm aware of the arguments against multi-threaded file copying (disk thrashing etc) but that doesn't apply here as the source and destination are on SAN boxes and therefore the copies will be from/to multiple disks at once.
The files are usually fairly big so the entire copy can take several hours to complete. I'd like to implement multi-threading so that the CopyAFile method is launched on a separate thread for each file.
What is the best approach here and how to do I ensure the all the threads (i.e. copies) are complete? I guess I have the following options
- BackgroundWorkers
- ThreadPool
- Parallel Processing (.NET 4)
Oh and I'm aware of the arguments against multi-threaded file copying (disk thrashing etc) but that doesn't apply here as the source and destination are on SAN boxes and therefore the copies will be from/to multiple disks at once.
Code:
For Each kvp As KeyValuePair(Of String, String) In VMsPerDisk
CDrivePath = kvp.Key
VMName = kvp.Value
If Not CDrivePath Is Nothing Then
'Adapt the path to be the shadowed version of the path
VHDShadowPath = CDrivePath.ToUpper.Replace(CSVPath.ToUpper + "\", NextAvailableDriveLetter & "\")
WriteLog("Path of VHD to backup = " + VHDShadowPath)
'Try to delete any previous backups. This is done because, should we use SCVMM in the future, it is
'possible that VHDs could be moved from one CSV to another. That would mean that, should we NOT delete
'any previous backups, we could potentially have multiple backup copies of the same VHD
DeleteAnyPreviousBackups(BackupRoot, VMName)
'Create the root backup folder if it doesn't exist
Dim DestinationFolder As String = BackupFolder + "\" + VMName
CreateFolder(DestinationFolder)
'Create file information object
Dim fi As New FileInfo(CDrivePath)
'Copy the VHD from the shadow copy to the backup folder
CopyAFile(VHDShadowPath, DestinationFolder + "\" + fi.Name)
End If
Next