The Order of the Stick: Utterly Dwarfed
The Order of the Stick: Utterly Dwarfed - Coming in December and available for pre-order now
Results 1 to 8 of 8
  1. - Top - End - #1
    Troll in the Playground
     
    BardGuy

    Join Date
    Jan 2009

    Default python help (copying files from one directory to another)

    I already solved this in SAS, but I'm trying to learn Python (within Anaconda) so I wanted to ask around. From some Googling, I got some code written but it doesn't work.

    My google results are saved as comments in the code.
    I have a set of CSV files that I need to distribute to school districts. My goal is to move the files at O:\data\myfiles to a sever with a folder structure like \\webftp\districts\Acct\<DistrictCode>\<DistrictCo de>000 for each district, where DistrictCode is a 4 digit number possibly with leading zeroes (e.g., 0900, 1000, 1100, etc.).

    The DistrictCode is embedded in the CSV files I'm moving, and I figured out how to get it out, but the actual transfer of files isn't happening. I'm designing the output directory correctly (as confirmed by the commented out print(output) line of code).

    Also, do you need multiple backslashes to make true backslashes (e.g., // means /)?
    Note that where I'm copying data to starts with two slashes instead of just one.

    Code:
    import shutil
    import os
    #https://stackoverflow.com/questions/25070614/python-moving-multiple-files-from-one-folder-to-the-other-based-on-text-characte
    #https://guide.freecodecamp.org/python/is-there-a-way-to-substring-a-string-in-python/
    
    dest1='O:\\data\\myfiles'
    dest2='\\\webftp\\districts\\Acct\\'
    
    files=os.listdir(dest1)
    for f in files:
        DistrictCode=f[9:13]
        output=dest2+DistrictCode+'\\'+DistrictCode+'000'
        #print(output)
        shutil.move(f,output)

  2. - Top - End - #2
    Orc in the Playground
     
    MindFlayer

    Join Date
    Feb 2015

    Default Re: python help (copying files from one directory to another)

    It doesn't work? What happens when you try? Do you get an error message?

    And, did the directories \\webftp\districts\Acct\<DistrictCode> already exist when you ran it?

  3. - Top - End - #3
    Troll in the Playground
     
    BardGuy

    Join Date
    Jan 2009

    Default Re: python help (copying files from one directory to another)

    The folder locations do exist.

    I tried re-running the code just now, and I got the message below. This seems to make sense as I had already written the file out there via SAS, so I guess this function only puts data out there; it doesn't overwrite a file already present.
    Code:
      File "C:\Users\<redacted>\anaconda3\lib\shutil.py", line 561, in move
        raise Error("Destination path '%s' already exists" % real_dst)
    
    Error: Destination path '\\webftp-0\0160\0160000\district_0160__DistName.csv' already exists
    So I changed dest2 to a dummy folder I just made that fits the same schema.

    Then I re-ran and got the same error as before.
    Code:
      File "C:\Users\<redacted>\anaconda3\lib\shutil.py", line 120, in copyfile
        with open(src, 'rb') as fsrc:
    
    FileNotFoundError: [Errno 2] No such file or directory: 'district_0160__DistName.csv'

    From the error message, it looks like it's trying to copy the csv to a folder named "district_0160__DistName.csv". When, if ever, should the name of the csv be present in the shutil.move call?

  4. - Top - End - #4
    Firbolg in the Playground
     
    Jasdoif's Avatar

    Join Date
    Mar 2007
    Location
    Oregon, USA

    Default Re: python help (copying files from one directory to another)

    Quote Originally Posted by JeenLeen View Post
    Then I re-ran and got the same error as before.
    Code:
      File "C:\Users\<redacted>\anaconda3\lib\shutil.py", line 120, in copyfile
        with open(src, 'rb') as fsrc:
    
    FileNotFoundError: [Errno 2] No such file or directory: 'district_0160__DistName.csv'

    From the error message, it looks like it's trying to copy the csv to a folder named "district_0160__DistName.csv".
    Actually, it's getting the error trying to open the source file (notice "fsrc" in the traceback). You'd want the full path to the file as the input, not just filename (which f is).

    Personally, I'd recommend using os.path.join for the path manipulations:
    Code:
    import os.path
    # Your stuff
    
    for thisFilename in files:
        DistrictCode=thisFilename[9:13]
        inputPath=os.path.join(dest1,thisFilename) # ...dest1 and dest2 instead of srcFolder and dstFolder?
        outputPath=os.path.join(dest2,DistrictCode,(DistrictCode+'000'))
        print("Input Path: {0}\nOutput Path: {1}\n\n".format(inputPath, outputPath))
        # Do real stuff here, obviously
    Last edited by Jasdoif; 2019-10-09 at 10:43 AM. Reason: BACK. TraceBACK. Gah!
    Feytouched Banana eldritch disciple avatar by...me!

    The Index of the Giant's Comments VI―Making Dogma from Zapped Bananas

  5. - Top - End - #5
    Troll in the Playground
     
    BardGuy

    Join Date
    Jan 2009

    Default Re: python help (copying files from one directory to another)

    Cool and thanks.

    Is there an easy modification to tell it to create the folder structure if it doesn't already exist?

  6. - Top - End - #6
    Firbolg in the Playground
     
    Jasdoif's Avatar

    Join Date
    Mar 2007
    Location
    Oregon, USA

    Default Re: python help (copying files from one directory to another)

    Quote Originally Posted by JeenLeen View Post
    Is there an easy modification to tell it to create the folder structure if it doesn't already exist?
    I just noticed I can link directly to functions in the official documentation....os.makedirs will make an empty directory structure for a path, and if you give it "exist_ok=True" as a parameter it won't give an error if the folder already exists; you should be able to call it for outputPath that way to automatically get the folder structure (and not have to explicitly check whether it already exists). And conveniently you're already importing os, so you should just need to add a line before the move:
    Code:
    os.makedirs(outputPath, exist_ok=True)
    Feytouched Banana eldritch disciple avatar by...me!

    The Index of the Giant's Comments VI―Making Dogma from Zapped Bananas

  7. - Top - End - #7
    Troll in the Playground
     
    BardGuy

    Join Date
    Jan 2009

    Default Re: python help (copying files from one directory to another)

    Thank you. It's working great now. My first work-operational Python program. Feels good!

    I also learned that what I really want was .copy instead of .move!

  8. - Top - End - #8
    Firbolg in the Playground
     
    Jasdoif's Avatar

    Join Date
    Mar 2007
    Location
    Oregon, USA

    Default Re: python help (copying files from one directory to another)

    Oh, almost forgot!

    Quote Originally Posted by JeenLeen View Post
    Also, do you need multiple backslashes to make true backslashes (e.g., // means /)?
    Note that where I'm copying data to starts with two slashes instead of just one.
    Assuming you meant to type \ (backslash) instead of / ((forward) slash)....

    Normally yes; as is the case in many languages, a backslash is how you reference special characters in a string ("\n" for newline, "\t" for tab, "\'" and '"' for quote characters that could clash with the string delimiter and/or your eyes, etc. etc.) so a literal backslash is treated as a special character ("") so you can still use a backslash in a string.

    Python does have a "raw string notation" for string literals which disables nearly all of that, just put an "r" in front of the opening string delimiter....While it's fine for most cases, it interacts rather unintuitively with the string delimiter; so it looks weird for a string that ends with a backslash...but since you're using os.path.join, you don't need to end dest2 with a backslash (it's the main reason I recommend it; "wait, did I end this string with a path separator or not" is both critical to proper functioning and a giant pain to handle, so having a function take care of all of that is very nice/productive). So you could do this:

    Code:
    dest1=r'O:\data\myfiles'
    dest2=r'\\webftp\districts\Acct'
    Feytouched Banana eldritch disciple avatar by...me!

    The Index of the Giant's Comments VI―Making Dogma from Zapped Bananas

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •