question

AkshaySolanki-2006 avatar image
0 Votes"
AkshaySolanki-2006 asked MotoX80 answered

How to copy folder/subfolders from source to destination using csv in powershell

Hello,

I am a new to Powershell and I have been given a task to copy all folders/subfolders files from source to destination using csv file in powershell.

I have managed to pull files from source to one destination but what if I need to copy the same file structure (including folder/subfolder) from source to destination (creation of same folder/subfolder) where no folder structure is created prior using csv and powershell?

This is the script below I used to copy files from source to destination but how about creating a similar folders and subfolders in destination with this.

CSV:

path
C:\folder1
C:\folder3
C:\folder5

Script:
Import-Csv C:\Scripts\source.csv | ForEach-Object{Get-ChildItem -Path $_.path |
Sort-Object -Property CreationTime -Descending |
Copy-Item -Destination 'C:\testfolder' -recurse -Force -Verbose}


Thanks in advance
Akshay

windows-server-powershell
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MotoX80 avatar image
1 Vote"
MotoX80 answered SolankiAkshaykumar-7685 edited

There is no need to use Get-ChildItem or Sort-Object. Just copy the folder and use -recurse to process the subfolders.

Try this.

 Import-Csv C:\Scripts\source.csv | ForEach-Object{Copy-Item -Path $_.path   c:\testfolder  -Recurse}
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

That worked.!
Thanks a lot :)

How can I add log file for this copy? and
Will it work with huge number of files as well?
for example we have 4000-5000 number of files to copy from one folder to another

0 Votes 0 ·
MotoX80 avatar image
0 Votes"
MotoX80 answered SolankiAkshaykumar-7685 commented

How can I add log file for this copy?

You could use the -Verbose switch to get a list of what was copied, but with thousands of files, that be a bit much.

 Import-Csv C:\Temp\Files.csv | ForEach-Object {
     "{0} Processing {1}" -f (get-date), $_.path  
     Copy-Item -Path $_.path   c:\test  -Recurse -force  -Verbose
     "{0} Processing Complete." -f (get-date)
 }


Will it work with huge number of files as well?


I have not done any performance tests on Copy-Item. Normally with a huge number of files I would use Robocopy with the /mir switch. That way if the copy process was interrupted, when you rerun it, robocopy with recognize that files already exist in the destination folder and not recopy them. And if a file were deleted in the source, it also get deleted in the destination. That's the "mirror" part. Robocopy is also multithreaded, so it should run fast. It also produces nice output that can be saved in a log with the /log switch.

 Import-Csv C:\Temp\Files.csv | ForEach-Object {
     $fldr = Get-Item $_.path                             # So we can get the name. 
     $dest = "c:\test\" + $fldr.name + "\"                # Build destination folder name 
     Robocopy.exe $_.path   $dest /mir /l                 # /l will list files/folders. Remove /l to do the actual copy 
 }





· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

HI, I tried the robocopy script and it works but it can't get the subfolders.
I used /e but still it doesn't copy subfolders to destination

0 Votes 0 ·
MotoX80 avatar image
0 Votes"
MotoX80 answered MotoX80 commented

I tried the robocopy script and it works but it can't get the subfolders.

I can't help you if you don't show any error messages or the command output.

It works for me.

 PS C:\> Import-Csv C:\Temp\Files.csv | ForEach-Object {
     "Processing {0}" -f $_.path
     $fldr = Get-Item $_.path                             # So we can get the name. 
     $dest = "c:\test\" + $fldr.name + "\"                # Build destination folder name 
     Robocopy.exe $_.path   $dest /mir /l                 # /l will list files/folders. Remove /l to do the actual copy 
 }
 Processing C:\Temp\foo1
    
 -------------------------------------------------------------------------------
    ROBOCOPY     ::     Robust File Copy for Windows                              
 -------------------------------------------------------------------------------
    
   Started : Friday, September 17, 2021 10:47:59 AM
    Source : C:\Temp\foo1\
      Dest : c:\test\foo1\
    
     Files : *.*
            
   Options : *.* /L /S /E /DCOPY:DA /COPY:DAT /PURGE /MIR /R:1000000 /W:30 
    
 ------------------------------------------------------------------------------
    
       New Dir          3    C:\Temp\foo1\
         New File              1111    foo1.sln
         New File               173    MyZippedFile.zip
         New File                92    SomeTextFile.txt
       New Dir          0    C:\Temp\foo1\.vs\
       New Dir          0    C:\Temp\foo1\.vs\foo1\
       New Dir          1    C:\Temp\foo1\.vs\foo1\v15\
         New File             47104    .suo
       New Dir          5    C:\Temp\foo1\foo1\
         New File               188    App.config
         New File              5146    foo1.vbproj
         New File              2207    Form1.Designer.vb
         New File              5817    Form1.resx
         New File               892    Form1.vb
       New Dir          0    C:\Temp\foo1\foo1\bin\
       New Dir          4    C:\Temp\foo1\foo1\bin\Debug\
         New File             14336    foo1.exe
         New File               188    foo1.exe.config
         New File             42496    foo1.pdb
         New File               651    foo1.xml
       New Dir          7    C:\Temp\foo1\foo1\My Project\
         New File              1486    Application.Designer.vb
         New File               510    Application.myapp
         New File              1148    AssemblyInfo.vb
         New File              2771    Resources.Designer.vb
         New File              5612    Resources.resx
         New File              2971    Settings.Designer.vb
         New File               279    Settings.settings
       New Dir          0    C:\Temp\foo1\foo1\obj\
       New Dir         11    C:\Temp\foo1\foo1\obj\Debug\
         New File              1443    DesignTimeResolveAssemblyReferences.cache
         New File              6906    DesignTimeResolveAssemblyReferencesInput.cache
         New File             14336    foo1.exe
         New File               180    foo1.Form1.resources
         New File             42496    foo1.pdb
         New File               180    foo1.Resources.resources
         New File                42    foo1.vbproj.CoreCompileInputs.cache
         New File               570    foo1.vbproj.FileListAbsolute.txt
         New File              1012    foo1.vbproj.GenerateResource.cache
         New File             10411    foo1.vbprojAssemblyReference.cache
         New File               651    foo1.xml
       New Dir          1    C:\Temp\foo1\foo1\obj\Debug\TempPE\
         New File              6144    My Project.Resources.Designer.vb.dll
       New Dir          1    C:\Temp\foo1\foo1\v15\
         New File             47104    .suo
       New Dir          0    C:\Temp\foo1\foo1\v15\Server\
       New Dir          4    C:\Temp\foo1\foo1\v15\Server\sqlite3\
         New File                 0    db.lock
         New File              4096    storage.ide
         New File             32768    storage.ide-shm
         New File             2.7 m    storage.ide-wal
    
 ------------------------------------------------------------------------------
    
                Total    Copied   Skipped  Mismatch    FAILED    Extras
     Dirs :        14        14         0         0         0         0
    Files :        37        37         0         0         0         0
    Bytes :    2.99 m    2.99 m         0         0         0         0
    Times :   0:00:00   0:00:00                       0:00:00   0:00:00
    Ended : Friday, September 17, 2021 10:47:59 AM
    
 Processing C:\Temp\foo2
    
 -------------------------------------------------------------------------------
    ROBOCOPY     ::     Robust File Copy for Windows                              
 -------------------------------------------------------------------------------
    
   Started : Friday, September 17, 2021 10:47:59 AM
    Source : C:\Temp\foo2\
      Dest : c:\test\foo2\
    
     Files : *.*
            
   Options : *.* /L /S /E /DCOPY:DA /COPY:DAT /PURGE /MIR /R:1000000 /W:30 
    
 ------------------------------------------------------------------------------
    
       New Dir          1    C:\Temp\foo2\
         New File             70793    PALFunctions.vb
    
 ------------------------------------------------------------------------------
    
                Total    Copied   Skipped  Mismatch    FAILED    Extras
     Dirs :         1         1         0         0         0         0
    Files :         1         1         0         0         0         0
    Bytes :    69.1 k    69.1 k         0         0         0         0
    Times :   0:00:00   0:00:00                       0:00:00   0:00:00
    Ended : Friday, September 17, 2021 10:47:59 AM

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

So it did worked for me. but I guess I am also getting parent folder even when I only type subfolder in the path
And using this I getting all the folders and subfolders instead of getting few subfolders (substructure1 and substructure3) and for mentioned path in csv
CSV:
133185-capture3.png



Code:

  Import-Csv C:\Scripts\source.csv | ForEach-Object {
      "Processing {0}" -f $_.path
      $fldr = Get-Item $_.path                             
      $dest = "c:\testfolder\" + $fldr.name + "\"                 
      Robocopy.exe $_.path   $dest  /e             
  }


Just for learning, what ""Processing {0}" -f $_.path" is for? I get the rest of the code but dont understand that

0 Votes 0 ·
capture3.png (9.4 KiB)

For some reason I am not able to paste the script process here as it is exceeding 1600 character limit even though I paste it as code

0 Votes 0 ·
MotoX80 avatar image MotoX80 SolankiAkshaykumar-7685 ·

Your initial question only shows one level of folders.

133201-capture.jpg


If you want to copy C:\Source\Folder1\MySubfolder to C:\Destination then you need to modify the script depending on if you want the end result to be in C:\Destination\Folder1\MySubfolder or C:\Destination\MySubfolder

I don't know what all is in your CSV file. Does every entry start with C:\Source or do you have different source folders?

As written the script should copy the subfolder to C:\Destination\MySubfolder. Did it do that?

Did you really want it in C:\Destination\Folder1\MySubfolder?







0 Votes 0 ·
capture.jpg (15.1 KiB)
MotoX80 avatar image
0 Votes"
MotoX80 answered

Just for learning, what ""Processing {0}" -f $_.path" is for?


It's just one way to format a string and insert variables into the text. That way we can see what the content of the variable is. See the section on Format String.

https://docs.microsoft.com/en-us/powershell/scripting/learn/deep-dives/everything-about-string-substitutions?view=powershell-5.1

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.