Masked (hidden) failures copying large folder with File Explorer











up vote
0
down vote

favorite












When I start copying a very large folder (say 40 GB) in Windows 7 File Explorer, the rate of progress indicates that it may take hours to complete. So I leave it overnight, only to discover in the morning that it still claims to be trying to copy (it has failed but still appears to be in progress).



Is there a way to copy a large folder with high confidence that it will be successfully completed, or at least that it will conclude and display an error message if there is a problem?










share|improve this question
























  • Last time I used Windows, it didn't even know if the destination had sufficient space for the file until it returned a 'disk full' error, half way through. Hopefully that has improved since Win7... but ...
    – Tetsujin
    Nov 21 at 19:30












  • @Tetsujin it really is a pain to get a large folder to copy via file explorer. This issue is not acknoledged too well. Using File explorer for large folders should be a no no to put it blunetly.
    – zar
    Nov 21 at 19:53






  • 1




    You've got a legit question buried in this, but it's worded in an off-topic way. It invites discussion, invites dispute about whether File Explorer is actually problematic, and asks for opinion on what's best. I'm voting to close on that basis, but will retract the vote if you can improve the question. Just state the problem you're facing and ask for a solution to it.
    – fixer1234
    Nov 24 at 1:16










  • @fixer1234 I have changed the title since.
    – zar
    Nov 24 at 16:14










  • Copying from one drive to another drive will be much faster than copying to the same drive. When copying to the same drive, the drive needs to keep finding, and moving between, the source and destination locations. If the folder contains a huge number of tiny files, it also needs to store directory information for each one. So that will take much longer than copying a few extremely large files. Huge folders won't be copied instantaneously, but 40 GB shouldn't take hours, even under the worst conditions (unless you are doing another drive-intensive, (cont'd)
    – fixer1234
    Nov 24 at 23:52















up vote
0
down vote

favorite












When I start copying a very large folder (say 40 GB) in Windows 7 File Explorer, the rate of progress indicates that it may take hours to complete. So I leave it overnight, only to discover in the morning that it still claims to be trying to copy (it has failed but still appears to be in progress).



Is there a way to copy a large folder with high confidence that it will be successfully completed, or at least that it will conclude and display an error message if there is a problem?










share|improve this question
























  • Last time I used Windows, it didn't even know if the destination had sufficient space for the file until it returned a 'disk full' error, half way through. Hopefully that has improved since Win7... but ...
    – Tetsujin
    Nov 21 at 19:30












  • @Tetsujin it really is a pain to get a large folder to copy via file explorer. This issue is not acknoledged too well. Using File explorer for large folders should be a no no to put it blunetly.
    – zar
    Nov 21 at 19:53






  • 1




    You've got a legit question buried in this, but it's worded in an off-topic way. It invites discussion, invites dispute about whether File Explorer is actually problematic, and asks for opinion on what's best. I'm voting to close on that basis, but will retract the vote if you can improve the question. Just state the problem you're facing and ask for a solution to it.
    – fixer1234
    Nov 24 at 1:16










  • @fixer1234 I have changed the title since.
    – zar
    Nov 24 at 16:14










  • Copying from one drive to another drive will be much faster than copying to the same drive. When copying to the same drive, the drive needs to keep finding, and moving between, the source and destination locations. If the folder contains a huge number of tiny files, it also needs to store directory information for each one. So that will take much longer than copying a few extremely large files. Huge folders won't be copied instantaneously, but 40 GB shouldn't take hours, even under the worst conditions (unless you are doing another drive-intensive, (cont'd)
    – fixer1234
    Nov 24 at 23:52













up vote
0
down vote

favorite









up vote
0
down vote

favorite











When I start copying a very large folder (say 40 GB) in Windows 7 File Explorer, the rate of progress indicates that it may take hours to complete. So I leave it overnight, only to discover in the morning that it still claims to be trying to copy (it has failed but still appears to be in progress).



Is there a way to copy a large folder with high confidence that it will be successfully completed, or at least that it will conclude and display an error message if there is a problem?










share|improve this question















When I start copying a very large folder (say 40 GB) in Windows 7 File Explorer, the rate of progress indicates that it may take hours to complete. So I leave it overnight, only to discover in the morning that it still claims to be trying to copy (it has failed but still appears to be in progress).



Is there a way to copy a large folder with high confidence that it will be successfully completed, or at least that it will conclude and display an error message if there is a problem?







windows-7 windows windows-explorer file-transfer






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 24 at 23:45









Scott

15.5k113789




15.5k113789










asked Nov 21 at 19:19









zar

4281615




4281615












  • Last time I used Windows, it didn't even know if the destination had sufficient space for the file until it returned a 'disk full' error, half way through. Hopefully that has improved since Win7... but ...
    – Tetsujin
    Nov 21 at 19:30












  • @Tetsujin it really is a pain to get a large folder to copy via file explorer. This issue is not acknoledged too well. Using File explorer for large folders should be a no no to put it blunetly.
    – zar
    Nov 21 at 19:53






  • 1




    You've got a legit question buried in this, but it's worded in an off-topic way. It invites discussion, invites dispute about whether File Explorer is actually problematic, and asks for opinion on what's best. I'm voting to close on that basis, but will retract the vote if you can improve the question. Just state the problem you're facing and ask for a solution to it.
    – fixer1234
    Nov 24 at 1:16










  • @fixer1234 I have changed the title since.
    – zar
    Nov 24 at 16:14










  • Copying from one drive to another drive will be much faster than copying to the same drive. When copying to the same drive, the drive needs to keep finding, and moving between, the source and destination locations. If the folder contains a huge number of tiny files, it also needs to store directory information for each one. So that will take much longer than copying a few extremely large files. Huge folders won't be copied instantaneously, but 40 GB shouldn't take hours, even under the worst conditions (unless you are doing another drive-intensive, (cont'd)
    – fixer1234
    Nov 24 at 23:52


















  • Last time I used Windows, it didn't even know if the destination had sufficient space for the file until it returned a 'disk full' error, half way through. Hopefully that has improved since Win7... but ...
    – Tetsujin
    Nov 21 at 19:30












  • @Tetsujin it really is a pain to get a large folder to copy via file explorer. This issue is not acknoledged too well. Using File explorer for large folders should be a no no to put it blunetly.
    – zar
    Nov 21 at 19:53






  • 1




    You've got a legit question buried in this, but it's worded in an off-topic way. It invites discussion, invites dispute about whether File Explorer is actually problematic, and asks for opinion on what's best. I'm voting to close on that basis, but will retract the vote if you can improve the question. Just state the problem you're facing and ask for a solution to it.
    – fixer1234
    Nov 24 at 1:16










  • @fixer1234 I have changed the title since.
    – zar
    Nov 24 at 16:14










  • Copying from one drive to another drive will be much faster than copying to the same drive. When copying to the same drive, the drive needs to keep finding, and moving between, the source and destination locations. If the folder contains a huge number of tiny files, it also needs to store directory information for each one. So that will take much longer than copying a few extremely large files. Huge folders won't be copied instantaneously, but 40 GB shouldn't take hours, even under the worst conditions (unless you are doing another drive-intensive, (cont'd)
    – fixer1234
    Nov 24 at 23:52
















Last time I used Windows, it didn't even know if the destination had sufficient space for the file until it returned a 'disk full' error, half way through. Hopefully that has improved since Win7... but ...
– Tetsujin
Nov 21 at 19:30






Last time I used Windows, it didn't even know if the destination had sufficient space for the file until it returned a 'disk full' error, half way through. Hopefully that has improved since Win7... but ...
– Tetsujin
Nov 21 at 19:30














@Tetsujin it really is a pain to get a large folder to copy via file explorer. This issue is not acknoledged too well. Using File explorer for large folders should be a no no to put it blunetly.
– zar
Nov 21 at 19:53




@Tetsujin it really is a pain to get a large folder to copy via file explorer. This issue is not acknoledged too well. Using File explorer for large folders should be a no no to put it blunetly.
– zar
Nov 21 at 19:53




1




1




You've got a legit question buried in this, but it's worded in an off-topic way. It invites discussion, invites dispute about whether File Explorer is actually problematic, and asks for opinion on what's best. I'm voting to close on that basis, but will retract the vote if you can improve the question. Just state the problem you're facing and ask for a solution to it.
– fixer1234
Nov 24 at 1:16




You've got a legit question buried in this, but it's worded in an off-topic way. It invites discussion, invites dispute about whether File Explorer is actually problematic, and asks for opinion on what's best. I'm voting to close on that basis, but will retract the vote if you can improve the question. Just state the problem you're facing and ask for a solution to it.
– fixer1234
Nov 24 at 1:16












@fixer1234 I have changed the title since.
– zar
Nov 24 at 16:14




@fixer1234 I have changed the title since.
– zar
Nov 24 at 16:14












Copying from one drive to another drive will be much faster than copying to the same drive. When copying to the same drive, the drive needs to keep finding, and moving between, the source and destination locations. If the folder contains a huge number of tiny files, it also needs to store directory information for each one. So that will take much longer than copying a few extremely large files. Huge folders won't be copied instantaneously, but 40 GB shouldn't take hours, even under the worst conditions (unless you are doing another drive-intensive, (cont'd)
– fixer1234
Nov 24 at 23:52




Copying from one drive to another drive will be much faster than copying to the same drive. When copying to the same drive, the drive needs to keep finding, and moving between, the source and destination locations. If the folder contains a huge number of tiny files, it also needs to store directory information for each one. So that will take much longer than copying a few extremely large files. Huge folders won't be copied instantaneously, but 40 GB shouldn't take hours, even under the worst conditions (unless you are doing another drive-intensive, (cont'd)
– fixer1234
Nov 24 at 23:52










1 Answer
1






active

oldest

votes

















up vote
0
down vote













Using windows file explorer to copy a large folder is, indeed, not very helpful for two reasons.




  • The main reason is that the file copy may fail for one reason or another, but it never returns and it will appear as if copying is still in progress where, in fact, it'ts just stuck. This can waste a lot of time of user.

  • If it failed due to error, it doesn't tell why it failed, what was the problem.


For example a file copy may fail because amid 40 GB data, there was one invalid shortcut where the target was missing. This will fail the copy but user will neer know and it will appear as if the copying is still in progress. There could be other reasons of failures that we we wouldn't know.



Solutions:




  1. It's best to copy large folders on command line and use `xcopy'. The benfit is that if it fails, it will return immediatly and you will know. That said it will still fail with above situation (invalid shortcut) but will error out correctly.

  2. The best Selection is to deploy a blind copy which doesn't care about the files but copy bits to bits. The best tools are scp or rsync but they are not readily available on Windows.


Using SCP on Windows



Install cygwin on Windows and use scpcommand from its shell the following way assuming to copy c:BigFolder to a USB drive F:BigFolderBackup/



scp -r /cygdrive/c/BigFolder/ /cygdrive/f/BigFolderBackup/


Where -r is for recurrsive copy which means copy all subfolders and their files.



Update



As noted by Scott in comments, since its copy on local machine so cp will do as well. Syntax is essentially identical.



Another scenario I came across is when a folder contain a file where I didn't had permssions to read. Our IT department doesn't allow users to run CCleaner utility so read permission is disabled on it.



I used cp this time and it performed nicely. It reported appropriate error on this file (didn't copy it) but didn't halt the copy operation and it continued with copying everything else. Another reason that this approach wins.






share|improve this answer























  • If you have Cygwin, and you're not copying from one machine to another, why no just use cp (instead of scp)?
    – Scott
    Nov 25 at 20:38










  • @Scott I guess that too, might be better in this case. It just didn't come on my radar since I had only used scp in the past
    – zar
    Nov 27 at 19:27











Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1377391%2fmasked-hidden-failures-copying-large-folder-with-file-explorer%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
0
down vote













Using windows file explorer to copy a large folder is, indeed, not very helpful for two reasons.




  • The main reason is that the file copy may fail for one reason or another, but it never returns and it will appear as if copying is still in progress where, in fact, it'ts just stuck. This can waste a lot of time of user.

  • If it failed due to error, it doesn't tell why it failed, what was the problem.


For example a file copy may fail because amid 40 GB data, there was one invalid shortcut where the target was missing. This will fail the copy but user will neer know and it will appear as if the copying is still in progress. There could be other reasons of failures that we we wouldn't know.



Solutions:




  1. It's best to copy large folders on command line and use `xcopy'. The benfit is that if it fails, it will return immediatly and you will know. That said it will still fail with above situation (invalid shortcut) but will error out correctly.

  2. The best Selection is to deploy a blind copy which doesn't care about the files but copy bits to bits. The best tools are scp or rsync but they are not readily available on Windows.


Using SCP on Windows



Install cygwin on Windows and use scpcommand from its shell the following way assuming to copy c:BigFolder to a USB drive F:BigFolderBackup/



scp -r /cygdrive/c/BigFolder/ /cygdrive/f/BigFolderBackup/


Where -r is for recurrsive copy which means copy all subfolders and their files.



Update



As noted by Scott in comments, since its copy on local machine so cp will do as well. Syntax is essentially identical.



Another scenario I came across is when a folder contain a file where I didn't had permssions to read. Our IT department doesn't allow users to run CCleaner utility so read permission is disabled on it.



I used cp this time and it performed nicely. It reported appropriate error on this file (didn't copy it) but didn't halt the copy operation and it continued with copying everything else. Another reason that this approach wins.






share|improve this answer























  • If you have Cygwin, and you're not copying from one machine to another, why no just use cp (instead of scp)?
    – Scott
    Nov 25 at 20:38










  • @Scott I guess that too, might be better in this case. It just didn't come on my radar since I had only used scp in the past
    – zar
    Nov 27 at 19:27















up vote
0
down vote













Using windows file explorer to copy a large folder is, indeed, not very helpful for two reasons.




  • The main reason is that the file copy may fail for one reason or another, but it never returns and it will appear as if copying is still in progress where, in fact, it'ts just stuck. This can waste a lot of time of user.

  • If it failed due to error, it doesn't tell why it failed, what was the problem.


For example a file copy may fail because amid 40 GB data, there was one invalid shortcut where the target was missing. This will fail the copy but user will neer know and it will appear as if the copying is still in progress. There could be other reasons of failures that we we wouldn't know.



Solutions:




  1. It's best to copy large folders on command line and use `xcopy'. The benfit is that if it fails, it will return immediatly and you will know. That said it will still fail with above situation (invalid shortcut) but will error out correctly.

  2. The best Selection is to deploy a blind copy which doesn't care about the files but copy bits to bits. The best tools are scp or rsync but they are not readily available on Windows.


Using SCP on Windows



Install cygwin on Windows and use scpcommand from its shell the following way assuming to copy c:BigFolder to a USB drive F:BigFolderBackup/



scp -r /cygdrive/c/BigFolder/ /cygdrive/f/BigFolderBackup/


Where -r is for recurrsive copy which means copy all subfolders and their files.



Update



As noted by Scott in comments, since its copy on local machine so cp will do as well. Syntax is essentially identical.



Another scenario I came across is when a folder contain a file where I didn't had permssions to read. Our IT department doesn't allow users to run CCleaner utility so read permission is disabled on it.



I used cp this time and it performed nicely. It reported appropriate error on this file (didn't copy it) but didn't halt the copy operation and it continued with copying everything else. Another reason that this approach wins.






share|improve this answer























  • If you have Cygwin, and you're not copying from one machine to another, why no just use cp (instead of scp)?
    – Scott
    Nov 25 at 20:38










  • @Scott I guess that too, might be better in this case. It just didn't come on my radar since I had only used scp in the past
    – zar
    Nov 27 at 19:27













up vote
0
down vote










up vote
0
down vote









Using windows file explorer to copy a large folder is, indeed, not very helpful for two reasons.




  • The main reason is that the file copy may fail for one reason or another, but it never returns and it will appear as if copying is still in progress where, in fact, it'ts just stuck. This can waste a lot of time of user.

  • If it failed due to error, it doesn't tell why it failed, what was the problem.


For example a file copy may fail because amid 40 GB data, there was one invalid shortcut where the target was missing. This will fail the copy but user will neer know and it will appear as if the copying is still in progress. There could be other reasons of failures that we we wouldn't know.



Solutions:




  1. It's best to copy large folders on command line and use `xcopy'. The benfit is that if it fails, it will return immediatly and you will know. That said it will still fail with above situation (invalid shortcut) but will error out correctly.

  2. The best Selection is to deploy a blind copy which doesn't care about the files but copy bits to bits. The best tools are scp or rsync but they are not readily available on Windows.


Using SCP on Windows



Install cygwin on Windows and use scpcommand from its shell the following way assuming to copy c:BigFolder to a USB drive F:BigFolderBackup/



scp -r /cygdrive/c/BigFolder/ /cygdrive/f/BigFolderBackup/


Where -r is for recurrsive copy which means copy all subfolders and their files.



Update



As noted by Scott in comments, since its copy on local machine so cp will do as well. Syntax is essentially identical.



Another scenario I came across is when a folder contain a file where I didn't had permssions to read. Our IT department doesn't allow users to run CCleaner utility so read permission is disabled on it.



I used cp this time and it performed nicely. It reported appropriate error on this file (didn't copy it) but didn't halt the copy operation and it continued with copying everything else. Another reason that this approach wins.






share|improve this answer














Using windows file explorer to copy a large folder is, indeed, not very helpful for two reasons.




  • The main reason is that the file copy may fail for one reason or another, but it never returns and it will appear as if copying is still in progress where, in fact, it'ts just stuck. This can waste a lot of time of user.

  • If it failed due to error, it doesn't tell why it failed, what was the problem.


For example a file copy may fail because amid 40 GB data, there was one invalid shortcut where the target was missing. This will fail the copy but user will neer know and it will appear as if the copying is still in progress. There could be other reasons of failures that we we wouldn't know.



Solutions:




  1. It's best to copy large folders on command line and use `xcopy'. The benfit is that if it fails, it will return immediatly and you will know. That said it will still fail with above situation (invalid shortcut) but will error out correctly.

  2. The best Selection is to deploy a blind copy which doesn't care about the files but copy bits to bits. The best tools are scp or rsync but they are not readily available on Windows.


Using SCP on Windows



Install cygwin on Windows and use scpcommand from its shell the following way assuming to copy c:BigFolder to a USB drive F:BigFolderBackup/



scp -r /cygdrive/c/BigFolder/ /cygdrive/f/BigFolderBackup/


Where -r is for recurrsive copy which means copy all subfolders and their files.



Update



As noted by Scott in comments, since its copy on local machine so cp will do as well. Syntax is essentially identical.



Another scenario I came across is when a folder contain a file where I didn't had permssions to read. Our IT department doesn't allow users to run CCleaner utility so read permission is disabled on it.



I used cp this time and it performed nicely. It reported appropriate error on this file (didn't copy it) but didn't halt the copy operation and it continued with copying everything else. Another reason that this approach wins.







share|improve this answer














share|improve this answer



share|improve this answer








edited Nov 29 at 19:29

























answered Nov 21 at 19:19









zar

4281615




4281615












  • If you have Cygwin, and you're not copying from one machine to another, why no just use cp (instead of scp)?
    – Scott
    Nov 25 at 20:38










  • @Scott I guess that too, might be better in this case. It just didn't come on my radar since I had only used scp in the past
    – zar
    Nov 27 at 19:27


















  • If you have Cygwin, and you're not copying from one machine to another, why no just use cp (instead of scp)?
    – Scott
    Nov 25 at 20:38










  • @Scott I guess that too, might be better in this case. It just didn't come on my radar since I had only used scp in the past
    – zar
    Nov 27 at 19:27
















If you have Cygwin, and you're not copying from one machine to another, why no just use cp (instead of scp)?
– Scott
Nov 25 at 20:38




If you have Cygwin, and you're not copying from one machine to another, why no just use cp (instead of scp)?
– Scott
Nov 25 at 20:38












@Scott I guess that too, might be better in this case. It just didn't come on my radar since I had only used scp in the past
– zar
Nov 27 at 19:27




@Scott I guess that too, might be better in this case. It just didn't come on my radar since I had only used scp in the past
– zar
Nov 27 at 19:27


















draft saved

draft discarded




















































Thanks for contributing an answer to Super User!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1377391%2fmasked-hidden-failures-copying-large-folder-with-file-explorer%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

AnyDesk - Fatal Program Failure

How to calibrate 16:9 built-in touch-screen to a 4:3 resolution?

QoS: MAC-Priority for clients behind a repeater