End user downloading an "archive" (Vanilla 2.3.1… I think)

2097

Hi, a forum where I've been writing for many years is closing down. I want to download every single post, or, at the very least, my own posts. I do not run this forum. Does anyone have good suggestions for how to do that?

Am thinking I'm just gonna go <code>wget --mirror</code> at the front page. That'll do the trick, right?

charrondev

You could try hitting the API endpoints maybe? If you use a script w/ a session cookie for your own user, all of the actions should obey the permissions of your user.

This would depend on what version of Vanilla that site is running though. The API here was initial rolled out in Vanilla 2.5 and expanded greatly in the last couple of years.

Alternatively you could treat it as an RSS feed. Appending .rss to any category or the recent discussions page gives you an RSS feed for it. You could crawl that, and throw .json on the end of the discussions themselves to get their contents if APIv2 is not available on that site.

R_J

You can try to threaten them with the GDPR and ask them to send you the data.

You can use some web scraper and download the complete forum 😲

You can ask them kindly to run the following sql and send you the result (for GDN_Discussion and GDN_Comment):

SELECT DateInserted, Name, Body, Format FROM GDN_Discussion WHERE InsertUserID = your-user-id;

Look that:

https://open.vanillaforums.com/profile/discussions/R_J.json?page=p1

...

https://open.vanillaforums.com/profile/discussions/R_J.json?page=p9

This is also possible with the comments. Find out how many pages you can download and start. Afterwards you should have all posts in JSON which can be parsed to give you the results.

2097

Big thanks for all the advice. It was clear, specific and useful, and I'm grateful. Another user beat me to the punch and scraped all of it (the forum admins were appreciative rather than apprehensive, which, kudos to them). He is a friend of mine so I have the data now. Doomsday averted!