Downloading a complete, searchable backup of one of this of forum?

[richmond62]

It has occurred to me that a forum a bit like this one, might, suddenly, without any warning at all, vanish 'in a puff of smoke', as the company that supports that forum made certain software items they offered disappear exactly in that fashion.

That forum contains what we could term "the distilled wisdom" of quite a few folk over quite a long period, and a lot of that is still relevant to our daily lives.

I wondered if there is a way to download all that data into a searchable database of some sort.

[tperry2x]

Yes and No.
Doing so is perfectly possible, but doing so also has a few issues attached with it.
1. Is scraping their server the right thing to do?
2. Depending on how heavily / aggressively this was done, it could be seen as a denial of service attack on that server.
3. Are we absolutely sure we are even entitled to everything on there?
4. Why not find the snippets of useful information yourself, and either save them as PDFs or save the page as an archive from the browser?

Point 4, is probably the better way to go anyway, because from what I've seen of that forum - the majority of it is just conjecture, endless waffle, arguments, and in many places - just dodging around shortcomings without getting to the answer. There's a lot of chaff to get through.

No doubt there are useful nuggets of information buried within it (like Heather's stacks, widgets etc) - but most of it is spurious. Something that we are in danger of repeating on this forum too. Case in point, it's hard to find posts sometimes because you have to sort through unrelated posts that really don't have any relevance.

[richmond62]

Surely 'scraping' and copying are not the same thing.

Also: searching for stuff that one feels is useful now might mean not getting stuff one feels is useful after the whole thing has gone down the tubes.

Oh, and if you poke around in the Wayback repository, many many things, that with hindsight would be useful are not available.

[tperry2x]

So, inevitably - question number 5:

Why do you think they would pull their forum suddenly? Surely doing so would be a huge own-goal, akin to shooting themselves in the foot?

[richmond62]

Why do I feel that shooting themselves in the foot is a habit?

It also would seem that they don't feel any sort of debt (even if only a moral one) to people who have donated money to them over the years.

One day (which will never come) it might prove informative to discover exactly what they understand 'community' to mean.

[tperry2x]

Why do I feel that shooting themselves in the foot is a habit?

It also would seem that they don't feel any sort of debt (even if only a moral one) to people who have donated money to them over the years.

One day (which will never come) it might prove informative to discover exactly what they understand 'community' to mean.

I'm also downloading all the bug reports currently, so we have a snapshot of that, for when it 'goes away'.

[tperry2x]

I'm also downloading all the bug reports currently, so we have a snapshot of that, for when it 'goes away'.

Well, while I've been powered off the best part of this week - my computer hasn't.
It's been chugging away, scouring the waybackmachine (specifically LCCs bug tracker) - the thing that was taken away from us previously and put behind a login barrier (again).

The script saves each article as a PDF, then creates a card with a text-based version of it. Clicking on the strange bug character will open the PDF if you downloaded that.
Screenshot:

screenshot.png (366.6 KiB) Viewed 1701 times

Download:
(mega link as it's too big for here)

Why it's useful:
There's a bug here, shown in the PDF on the left, existing since v2.9.1. It spans all the community releases, and persists at least until 9.6.9 rc1.
I just picked this at random. There are 5000+ of them. This makes the dictionary edits look an easier (I didn't say 'easy') job.

Anyway, going back to bed now for another couple of days. I know how Hal9000 felt when Dave was pulling his memory chips out one at a time.

[OpenXTalkPaul]

Download:
(mega link as it's too big for here)

Why it's useful:
There's a bug here, shown in the PDF on the left, existing since v2.9.1. It spans all the community releases, and persists at least until 9.6.9 rc1.
I just picked this at random. There are 5000+ of them. This makes the dictionary edits look an easier (I didn't say 'easy') job.

Anyway, going back to bed now for another couple of days. I know how Hal9000 felt when Dave was pulling his memory chips out one at a time.

Oh nice, even sick in bed and you're working it! Nice, thanks!
There's a few times I've clicked on some link about a bug, from a forum or an old list post and it points to that bugzilla server and then I get frustrated. It will be useful to have our own offline copy of the data (and formulas and even some solutions proposed by community members who reported them).

[richmond62]

That is the occupational hazard of working in schools, especially at this time of year: children bearing "Tidings of Great Infections."

Get better quickly: all is forgiven.

[richmond62]

Editing the Dictionary is far, far easier than sorting out bugs in code: the worse thing is that it is incredibly boring: and because of that there is a real danger one loses focus (coffee, hot bath, bang head on wall, cuddle the cat, read trashy novel) and misses a salient point.

But as my computer programming skills are nowhere near as advanced as yours I shall keep chuntering along on the 'Bluebell Line'.

Having 'done' UNIX and MacOS, and fully occupied elsewhere, I shall start a 'go through' every text file at the start of January.

I also intend to write a few text files of my own: starting with some guff about SVG files, and Unicode. Both almost undocumented, and both extremely relevant.

[richmond62]

A bug that has existed from RR 2.9.1 to LC 9.6.9.

And those people probably look at themselves in the mirror each morning.

[tperry2x]

But as my computer programming skills are nowhere near as advanced as yours I shall keep chuntering along on the 'Bluebell Line'.

I'm only a hobbyist. Not a 'serious coder' (whatever a serious coder is).

People have said "why didn't you get a jab?" - I'd have loved to. I'm not an anti-vaxxer or anything. Whenever I'd gone to the GP, they'd given me times in about 2 weeks or times that I couldn't get out of work for. Well too late now, even though my appointment would have now been in 8 days from today.
The government have vaccination teams, driving around vaccinating the over 65 year olds. That's fine, and great - but there's no provision for those people working 9-5 (or 7-5) - we are all just supposed to get on with it.
What they should have done, is like a dentist, send you a text with:
"Your flu and covid vaccinations are booked in for [day] at [time]."
They'd then have a pre-arranged appointment for everyone (as they know each patient they have registered at the GP, they know their medical history, they know if they've had a jab or not). They'd then also know they had enough vaccines ordered for everyone.
But that's too bloody easy it seems.

A bug that has existed from RR 2.9.1 to LC 9.6.9.

There's quite a few instances of things like this, that have remained unfixed. I think these have just been stacking up to the point where they took the decision to recreate from afresh rather than patch LC classic.