Submitted by unlikemike123 t3_z6wvdf in explainlikeimfive
unlikemike123 OP t1_iy3hpkd wrote
Reply to comment by nmxt in eli5 How is computer memory deleted? by unlikemike123
So when I delete it I'm just saying "this cannot be accessed and has permission to be overwritten" but it's still "physically" there?
Persist_and_Resist t1_iy3hulw wrote
That is exactly yet, and that is why it can almost always be recovered up until the point in which it is written over.
boersc t1_iy3j0vp wrote
There is also the action of a 'destructive delete', where the file is actually overwritten with gibberish. This is done when you want to delete a file without ever being able to restore it. This could be the case when handling confidential information, or in case you plan to retire the PC and hard disk.
Basically, you're telling the system to do the 'overwriting' action immediately.
ElAdri1999 t1_iy48082 wrote
But if you can do the hammer delete then it's way better, since someone might be able to recover your stuff if not sufficiently deleted
[deleted] t1_iy4loxs wrote
[deleted]
CrazyMando t1_iy4nqq0 wrote
Sledgehammer
Pikassassin t1_iy4py1y wrote
Gigantic electromagnet
ElAdri1999 t1_iy4qk5j wrote
All, all is good.
My last dead HDD I removed the magnets(which are fucking strong as fuck btw) and spun the platter while scratching lines on it, when I finished I broke it just for the safety
yoshhash t1_iy4a8jw wrote
Isn't that the same thing as over writing with the same file name?
boersc t1_iy4apc9 wrote
Not perse. In that case you're erasing/reusing the ledger entry, but the 'new' file may get written on a wholly different location depending on how the OS works. But yes, it CAN work the same, if the new file overwrites the exact same spot as the old file. Most OSses work that way.
bob0979 t1_iy4blr4 wrote
Many drive cleaner tools will have a tool to wipe free space. They write over every bit that is not occupied with 'undeleted' data or stuff you're using. They replace everything not in use with new junk data that could just be 00000001 for every bit. This takes a fair bit of time but doing this a couple of times can remove any trace of what a specific bit says or even used to say.
If you save somethinn as 'file.name' and it's contents are 00000111 then delete it, it stays as 00000111, but if you delete it then wipe free space it changes that 00000111 to something useless.
Grezzo82 t1_iy5aoeu wrote
“A couple” is sometimes not enough, but a few more is considered secure enough for most contexts, though gov will often physically shred the disks to be sure.
I only know for sure with macOS, but I image isn’t this applies to all: the OS has a built in secure erase feature that will overwrite a whole disk enough times to be confident that the data is irrecoverable
Any-Broccoli-3911 t1_iy4fv5b wrote
No, even if it's the same filename, there's no guarantee that it will be at the same place. Also, if your new file is smaller, for example an empty file, it can only overwrite a tiny part of it if ever it's at the same place.
graebot t1_iy4q95g wrote
Does that work with SSDs? I thought they try to write in a new location each time to prolong life
Grezzo82 t1_iy5arcz wrote
SADs are much harder (perhaps impossible) to securely erase, you’re right
unlikemike123 OP t1_iy3i3q0 wrote
Ok, so why does my computer say that I now have more memory if it's still there? Shouldn't it technically occupy the same amount of space?
unlikemike123 OP t1_iy3i9q4 wrote
O wait, that space is now available to be taken over by new data?
Hyperspacehost t1_iy3wgqy wrote
It’s worth thinking of computer memory less like a resource that’s used up, like water being scooped out of a bucket and drank, and more like sand in a sandbox.
Sometimes it’s being used for something meaningful, like being a castle or making the walls of a big hole or having letters written in it, and sometimes it just lying around doing nothing, but there’s not any more or less of it.
You might draw a line around a bit and say, “my castle is here, don’t use anything inside the line”, and at the end of the day you rub the line out and go, “I’m all done now, so do whatever”. Maybe you come back and the castle is still there or maybe someone has knocked it down and built something else with it.
SordidSimpleton t1_iy4461p wrote
This is a really cool analogy, props
Persist_and_Resist t1_iy3inlp wrote
Yep, that is what it means.
Neuro0Cancer t1_iy3iplc wrote
Exactly. The "free" memory is just the available memory, not the empty memory.
In fact memory that has been overwritten can still be recovered.
That's why there is software that "clean" your disk if you want, and it takes a while because basically what it does is just overwrite over and over again the disk until recovery is almost undoable.
[deleted] t1_iy41j00 wrote
[deleted]
AdmirableOstrich t1_iy470c2 wrote
There isn't really (official) publicly available software to do this, but depending on the storage technology you can often tell not only the current binary state of a memory bit but also the previous one.
As an example, we might say a bit is on if it is at 5V and off at 0V. However, real bits have voltages that vary quite a bit and we just threshold at (say) 2.5V. The actual voltages depend on (among other things) the previous state of the bit. If AB denotes a previous state A and a current state B, you might have something like:
00 -> 0.5V 10 -> 0.9V 01 -> 4.1V 11 -> 4.5V
Note that is doesn't quite work like this. The main point is that with dedicated hardware you can try to read data that has been lightly overwritten. This is why tools intended to purge data will overwrite many times with random bit sequences to minimize the chance of data recovery.
Alternative_Log3012 t1_iy4kkgi wrote
You won’t be able to do these sort of voltage reads in the standard OS the computer is running. I’m assuming you will need to remove the drive and put it into dedicated forensic recovery hardware. This could be even more difficult if your SSDs are soldered on like it most modern ultra books.
tim36272 t1_iy439pq wrote
Programs like Recuva do it for you. In the basic mode it is just reading all the entries that are marked as deleted.
Alternative_Log3012 t1_iy4k6mr wrote
This is not true / not what the OC is asking
dale_glass t1_iy3ircn wrote
Yup.
Also, fun fact, computers lie to users all the time. Eg, say you're playing a game and save. The game with very high likelihood will tell you it has saved well before the computer actually did it, and resume playing.
Why? Because people hate waiting, and the computer can actually do the writing to disk while you keep playing without making you wait for everything to be done.
There's many, many such optimizations.
ThatGenericName2 t1_iy3vbv5 wrote
There's also the lie in the other direction. While I don't think it's practiced anymore, sometimes software would "fake load" in that the program simply just waits for an extra bit of time because people would think that a computer didn't do something properly if it worked too fast.
goclimbarock007 t1_iy41vuk wrote
I read an anecdote about a programmer that wrote some sort of script that worked too fast and didn't look like it actually did anything. He added sleep counters and a progress bar to make it seem like it was doing something. A year later he "optimized" the program by cutting the sleep timers in half.
ClownfishSoup t1_iy4mkcj wrote
This weird thing happens with cake and cookie mixes as well. You can easily make a mix that just requires adding water and then baking, but people didn't believe they would be any good, so manufacturers changed the ingredients so that the end user has to add oil, milk and eggs to the mix. ie; people were suspicious that the mix was too easy, so they dumbed it down.
Brian57831 t1_iy41vtz wrote
Until you crash shortly after saving and the file shows it's corrupted.
DeHackEd t1_iy3mcrz wrote
People often ask why deleting files is faster than creating files. The answer is because the bulk of the file's data is NOT erased, because that's considered wasted effort. When the new file comes along it will write over the data, completely unaware of what was once there. So actually blanking out the old data doesn't really accomplish anything and the "delete" is really fast by just dealing with the file listing paperwork.
...unless there's privacy concerns that warrant actually clearing the data, specifically because preventing undelete from being possible is the goal. This is sometimes called "secure delete". But if you're uninstalling a video game, who cares?
EcchiOli t1_iy3oori wrote
Mostly useless remark, quoting you,
> People often ask why deleting files is faster than creating files
That's the case for the most common operating systems. However, sometimes, with server hardware + software, it's working differently and deletion can take a long time, as with linux's ext4. It's still relatively fast, but much slower than the others nonetheless.
kucksdorfs t1_iy3uze3 wrote
Think of it as a table of content in a book. So when you "delete' a file, you just remove the table of content chapter marker, but not the pages. Then, when you need to write more data, you "write" to the pages then update the table of content.
Things can get more complicated when it comes to flash storage (ssd/flash drive) but only when it comes to overwriting the pages.
eotty t1_iy45ump wrote
Imagine a binder with a table of contents, inside the binder there are pages, when deleting a file you just erase the page number, the page is still in the binder but you no longer know where to look. (If you want to find the pages again, there is special software, akin to looking through the binder page by page)
Some documents are several pages long, so when you later write something to the binder and replaces the pages, you might have page 4,7,9 of your document left, thats when it cannot be recovered anymore.
Defleurville t1_iy3lxmr wrote
Just to clear up any confusion, the word memory is often used in confusing ways. Expensive working memory (RAM) is used to do stuff (like a kitchen counter), whereas cheaper storage memory (SSD, HDD) is used to keep stuff (like kitchen cabinets).
Current computers have working memory in the 16GB to 128GB range and storage memory in the hundreds to thousand of gigabytes (known as terabytes)
Phones and tablets (and video game consoles) all have working memory, but the numbers are basically unadvertised (~up to 4 GB). Their storage memory is widely advertised, but is often in the same 16-128GB range as a computer’s working memory, which adds to the confusion.
While technically these are all “memory”, when we speak of a computer’s memory, we usually mean working memory (RAM) — not the one freed up by deleting files. But when we speak of a phone’s memory, we usually mean storage memory (SSD)
Any-Broccoli-3911 t1_iy4gc16 wrote
Your free memory is the memory your computer is allowed to use, which includes any deleted files even if it's still there. Your computer doesn't care about the information in the free memory, but it's recoverable.
21_MushroomCupcakes t1_iy4olnj wrote
Stuff that's "marked for overwrite" isn't counted against total capacity.
I would just like to make the distinction that memory and storage are not the same thing, memory is your RAM and storage is your hard drive.
Many people are using them interchangeably, which will run aground if memory addresses ever come up.
clideb50 t1_iy4ece7 wrote
It’s also why it’s good practice to pull the drive out of the computer when throwing it away/recycling it, and then destroying the drive.
[deleted] t1_iy4empb wrote
[removed]
GoodTodd1970 t1_iy3iiu7 wrote
There are specific programs that allow file "shredding" or "secure deletion", which will overwrite the file location with junk data multiple times. The original file no longer exists and the space is free to use.
unlikemike123 OP t1_iy3imhn wrote
Oooh that's how they work, cool!
snozzberrypatch t1_iy3r50b wrote
There isn't really any such thing as "deleting" memory so that it becomes "empty". Individual memory locations can either be a 0 or a 1 at all times, there are no other options. You could write all zeros into the memory but that's still data. There is no way to "clear" a memory location such that it's neither a 0 nor a 1. That's why it's most efficient to delete a file by just freeing up those memory locations but not actually overwriting them, unless you're concerned about Russian spies getting your hard drive and restoring deleted data.
unlikemike123 OP t1_iy3ympg wrote
I am very concerned about Putin seeing my endless lists of goals that are never achieved tbh, he's a harsh critic
[deleted] t1_iy4c3bs wrote
[removed]
Defleurville t1_iy3k9vj wrote
Note that different types of storage handle things slightly differently.
Magnetic storage (HDD, floppies) cannot delete files, it can only overwrite them, so the space will never stand empty.
Flash storage (SSD or a memory stick) will actually delete the file (or parts of the file) at some point (but you won’t know when) and may leave space empty for a while.
Re-writable optical storage (DVD-R) will basically wipe disc contents when you erase it (but is really bad at deleting individual files).
RAM (there are some circumstances where you can delete a file from RAM, such as when using a RAM disk) will retain the file “inaccessibly” until it is either overwritten or the RAM loses power, which causes it to lose all contents.
Note that in every case, “inaccessible” just means harder to access. If the data is there, there are ways of reading it.
Kientha t1_iy3m2ka wrote
And this is why flash storage devices are a nightmare for forensic purposes. With magnetic storage, you can attach a write blocker and know the data on the drive won't change so you can compare hash values and verify nothing changed. Flash memory however will randomly change data so you can't rely on hash values to prove nothing has changed
Defleurville t1_iy3uvx0 wrote
Another precision for other readers, which I assume Kientha knows, is that when speaking of computers, “randomly” never means randomly, as computers are incapable of doing anything at random (even electronic slot machines aren’t random).
In some cases, in means “not in sequential order”: We don’t continue reading a dictionary from where we stopped last time, but we do go in looking for a specific word, not pop it open and read whatever’s on the page. In computer terms, despite not being at all random, this is called “random access”.
In other cases, it means “made to appear random to a human”. Computers can easily generate values where users can’t tell what it will pick next, but under identical circumstances (the “seed”) it will pick the same values in the same order every time. We generally “cheat” by incorporating the time (down to the millisecond) into generating the numbers, so it’s mostly unpredictable.
Changes in Flash data aren’t random, they’re 100% deterministic and predictable (knowing all the info): they’re just not readily predictable to a user, which is functionally basically the same.
squigs t1_iy3u6o8 wrote
Yup. Data recovery firms rely on this.
There are actually some fairly simply tools for most filesystems that will find deleted files.
It's the same with RAM. I think modern systems are more secure, but in the past, hackers were able to break into systems by allocating a large chunk of RAM and seeing if anything that looks like a password was there.
Zoso03 t1_iy4gh82 wrote
Think of it like a book. It's going to the table of contents and saying these pages are empty and can be used. Yes words are still on the pages but now you are free to erase what's there and write something new. This is why things can be recovered.
Destructive delete is deleting the table of contents entry then going to pages and erasing everything on those pages. Some methods will then write over the data with garbage several times. So with the same analogy it's erasing the page writing gibberish then erasing it so even the intends of the last letters are impossible to make out.
freecain t1_iy3rnv1 wrote
Exactly. If you ever get rid of a computer, it's worth spending the money on a program that will right over the files methodically with null files. Alternatively, just encrypt your device when you first set it up and you should be fine (though encryption could be broken at some point in the future).
TheSnarfles t1_iy3w28z wrote
Think of it like that mural the previous tenet painted on the wall at your apartment. The landlord just paints over it, but you could recover it if you really wanted to.
therealpigman t1_iy3wumy wrote
Some memory cards have the ability to write the deleted data with all 0s in the places, but that isn’t guaranteed behavior and usually takes more time and energy than allowing the old stuff to be overwritten
tweiss84 t1_iy4il9y wrote
This is why you can sometimes recover data from a corrupted or crashed hard drive. In computer forensics they find "deleted" files.
Not removed, just deallocated.
Terr_ t1_iy4rszf wrote
Imagine that computer RAM or a classic hard-drive are like a bunch of tiles an Othello/Reversi game board.
When you delete a picture, only the tiles that say "photo.jpg is inside tiles X to Y" are erased and flipped to be white-side up.
The actual tiles in spaces X to Y that had all of the pixels are usually left untouched, unless we have a paranoid reason to go through and change them.
In no case are any tiles being removed from the table--that would represent damage to the system.
Later those X-Y tiles may be changed, but usually because we've decided to put something new there.
Kriss3d t1_iy4to8g wrote
Yes. That's how undelete programs works.
Think of a drive as a book. You got the list in the first part with the pages where each chapter begins and ends and what the chapter is called.
That's in the computer called a partition table.
And instead of a chapter it's a file. But pretty much like that.
So if you want to delete a drive you don't delete the actual pages. You just clear the index page where the name of the chapter is.
So if there's a range of pages that have no chapter name then it means you could replace the chapter with a new chapter ( so to speak) and add a chapter name for it.
This makes formatting a big drive quick. You simply erase all the names in the index.
All the data is technically still there.. But because the index said it's empty, the computer will Just overwrite that part and add a new name in the index.
That's actually exactly how a drive on a computer works. Ans also why you can undelete files or drives if they haven't been overwritten.
PaxNova t1_iy3vssw wrote
On older hard drives with magnetic memory (not the fancy M.2 and flash stuff we have now), you had to defragment drives.
Data was stored magnetically and you had to move a magnetic reader head over it to recover the data. Because the data took up physical space, it was faster for hard drives to store related data close together. As you deleted some files, the remaining files stayed in the same physical place. To make things faster, you would copy your files every now and then to condense them. Bringing all those fragments together was "defragging."
In modern drives, it takes the same amount of time to access all parts of your hard drive, so defragging is no longer necessary.
Viewing a single comment thread. View all comments