minor programming rant

[NichG]

So that means the NULLs were not random. From the measurement techniques alone there are some bounds on what is detectable or not, which means that NULLs do carry some useful information. As they say, lack of news is good news.

Understanding those patterns however without going through the data personally might be not possible.

Yep. The other idea was to make a compressed representation that captures as much of the data as possible (e.g. can reconstruct it), but which makes it impossible to infer which fields were NULL just by looking at the representation. However, that necessarily reduces the information content of the points we do have more complete data for. For example, if the period and masks have some mutual information I, and the period has entropy H>I, I think the cleaned representation can only have H-I information about the original period (because otherwise you could infer the masks, since information is symmetric). So, it's tricky.

[caden_varn]

As for kicking yourself for not keeping the original version - this is why you develop/test in a different area to production, and only move live once all testing is complete. And preferably use a source code repository that keeps all your previous code versions.

[Radar]

As for kicking yourself for not keeping the original version - this is why you develop/test in a different area to production, and only move live once all testing is complete. And preferably use a source code repository that keeps all your previous code versions.

There are only two kinds of people: those who do backups and those who will be painfully pushed to the first category.

Separating development and production environments is just a sane way of having a stable version clients can actually use on a daily basis.

[halfeye]

There are only two kinds of people: those who do backups and those who will be painfully pushed to the first category.

Separating development and production environments is just a sane way of having a stable version clients can actually use on a daily basis.

I've seen it written: "If it's not backed up, it's not important".

[factotum]

I've seen it written: "If it's not backed up, it's not important".

My version is "If you're not backing it up, you don't care if you lose it.".

[halfeye]

My version is "If you're not backing it up, you don't care if you lose it.".

Fairer, but not quite so harsh, and maybe thus less memorable?

[EldritchWeaver]

You shouldn't just back up source code files, you should use git or similar software to be able to check changes over time.

[El'the Ellie]

In the topic of preserving our own computer mistakes, I'll share mine for the record.

If you're running an old script by hand (as I was, when trying to find the particular line that was causing issues), please please make sure you get the same environment that the script is actually run in.

I reached the line

sudo chmod -R 400 /$PATH_TO_DIR$DIR_NAME

Of course, PATH_TO_DIR and DIR_NAME are usually set at this point, but not in my shell! Yeah...

sudo chmod -R 400 /

[caden_varn]

In the topic of preserving our own computer mistakes, I'll share mine for the record.

If you're running an old script by hand (as I was, when trying to find the particular line that was causing issues), please please make sure you get the same environment that the script is actually run in.

I reached the line

sudo chmod -R 400 /$PATH_TO_DIR$DIR_NAME

Of course, PATH_TO_DIR and DIR_NAME are usually set at this point, but not in my shell! Yeah...

sudo chmod -R 400 /

.... Ouch.

Reminds me of a friend at university (a long while ago) who managed to write-protect his hard drive (the one with the boot sector). Not quite sure how he managed to do that, but as the write-protect switch was on the hard drive, and he couldn't write to it any more...

In the end we had to remove the hard drive and boot it up as a slave on another system to unset what he'd done.