Yahoo Groups Shutting Down - Dec 14, 2019 (UPDATED)

[morgandawn]

(Updated Oct 25, 2019)

Starting Oct 28, 2019 no new files (photos, documents, etc) will be allowed to be uploaded. Starting Dec 14, 2019, Yahoo Groups will be limited to emails only - functions like digest will be eliminated and all files and photos will be deleted. Admins will have limited tools and all groups will be invite only. And archives of existing messages will not longer be accessible.

What To Do?

I have a step by step guide here. The short version

1. Admins - back up your Groups and decide whether and where to move
2. Members - back up your Groups - don't leave this to the admins.
3. Everyone - consider documenting Yahoo Groups on Fanlore, help spread the word

The OTW has made an announcement. Their Open Doors preservation program is open to  providing storage of Yahoo Groups backups that are assembled by moderators and non-moderators alike. Details here.

Fans are gathering on a Discord Channel to chat and ask for and give help:  https://discord.gg/ek2xFyJ (updated invite)

Please follow the Yahoo Group Rescue Tumblr for updates

 

Comments

[shadowscast]

The free trial version of the software will only export 1000 messages as HTML, total. What it produces is are basic but fairly nice-looking HTML files with 50 messages per page; there's a clickable contents list at the top and the messages underneath. The files are generated with the names HtmlDigest1.html, HtmlDigest2.html, etc. (I can't remember if there was an option to alter the names before the exporter ran.)

You can also export all of the messages as one big text file by choosing the "pipe delimited html" option; that works without any limits in the free trial version. If you change the filename extension to .html instead of .txt afterwards, you can open it in a browser like Chrome or Firefox or whatever. It's a bit ugly, but easily human-readable. (Note: I'm speaking only from my own experience of doing it with the ~2500 messages from my one group.)

[morgandawn]

I've also been told you can change the extension of the big old txt file to .mbox and import it into Thunderbird. Have not tested it.

[morgandawn]

I have SQLiteStudio - I can navigate to the .db3 file and "Add a Database" but nothing opens. I may not understand MySQL enough to operate it. Keep in mind that my db3 contains many groups - not just one

[shadowscast]

Okay, I'm not sure how helpful I can be because I don't understand mySQL at all, and I don't even remember exactly what I did to get my database to open; I just poked around a bit and it magically worked.

Would it be helpful possibly if I sent you a screenshot of what mine looks like with the database open? Is there a way for me to do that? (Email, maybe?)

[morgandawn]

yes email is fine morgandawn@gmail.com

[doranwen]

I meant "can you do export to HTML with SQLiteStudio?" The message I commented on was you viewing the messages in SQLiteStudio so I asked about exporting in conjunction with that. SQLiteStudio says it can be done on the home page, but I haven't tried even installing it yet. I'm just backing up as much as I can right now and will worry about exporting after that. I don't have the $$ to pay for PGOffline so I was looking for an alternate way to deal with that.

EDIT: Replied to the wrong person, sorry!

[doranwen]

See my reply to morgandawn below clarifying what I was asking. (I commented on the wrong comment.)

[chrystalline]

Ooooh, it sounds like you and I have a lot in common! Maybe we should take it off morgandawn's comments, and make it a FanFicFare Focused Post.

[shadowscast]

Oh, whoops! Yeah, in my previous reply I thought you were asking about PG Offline.

I don't know about exporting messages to HTML with SQLiteStudio; I have not tried. The only thing I've done so far is discover that I can essentially grab the messages one-by-one by hand in HTML format (by opening them up one-by-one in the database and copy-pasting the text).

I should mention that I really don't know anything about SQLiteStudio, or SQL in general.

[shadowscast]

Okay, following understanding your question properly I went back and played around a little with SQLiteStudio and the answer turns out to be YES, you can export in HTML. I have now managed to produce a giant, reasonably tidy HTML file with every single message from my mailing group's history.

[shadowscast]

Here's how I did it:

I went to Tools->Export.

When it asks "What do you want to export?" I selected "A single table." Hit next.

Then I selected the correct database (the group I wanted to export, the .db3 file for which needed to have been previously opened in SQLiteStudio), and selected the table "group_message" from the drop-down box.

I kept all of the default ticky-boxes ticked under "options" in this window, and hit "next".

That brought me to the "Export File and Options" window.

Export format: HTML

Output: file (select your desired location to save it)

Export format options: I ticked all the boxes, including "Don't escape HTML characters" (which was not ticked by default).

Maximum number of characters per cell: change it to something ridiculously high so that long fic posts won't get cut off. I set it to 10000000.

Output format: Format document (new lines, indentation)

Click finish.

This resulted, as I said, in one giant and very readable giant HTML document with every single message ever sent to the group. (It's actually in a grid, with the date and subject line in cells next to the message contents.) The one with my group's 2500 messages is 14,781KB in size.

Whether there's a way to select only certain messages to output in the file is something I do not yet know. (But of course you could always open up the HTML file in an editor and start chopping bits out.)

[shadowscast]

Er, sorry for the spamming, but I'm learning as I go! I wanted to mention that I just discovered that the table produced in the way that I just described won't include the information about who sent each message in an obvious way. However the info is recoverable; there's a column called "person," and the number in the cell will identify the sender of the message (with the numbers being matched with names on a separate table called "person").

[doranwen]

No problem! I'm sure I'll come back here and poke at this when I get to exporting and all.

*headdesk* I just realized… it would've been smarter for me to create a new db3 for each group. I've got multiple groups saved into the same db3 file. Is it still possible to separate those out or will it smush all of those into a single export? (I will totally separate them from now on!)

And I do know a bit of SQL now since I taught myself (with copious amounts of help from kind souls on the OpenOffice forum) how to create a database this summer. I'm still not super experienced in creating queries and whatnot, but those tables can be easily matched up that way. You do a SELECT and grab the name column from the person table along with the data from the table of the messages, and you have to add a line that goes WHERE "person"."personID" = "messages"."personID" essentially (I don't know what the actual table info is there so I just made up names so it's clear what's going on). But I did all my SQL queries with a LibreOffice Base files, so this might be a bit different, we'll see.

[shadowscast]

Is it still possible to separate those out or will it smush all of those into a single export?

Since I've only downloaded the one group, I can't really say.  When I was doing the export it looked as though, if I had more than one database, I could choose to select only one of them.  What I don't know is whether the program would generate (read? differentiate?) multiple databases from one .db3 file if you'd downloaded multiple groups.

[doranwen]

I've had a couple of the groups I'm backing up start downloading messages later than the earliest. In one case, the earliest it would start downloading was 16891 - messages from February 2003 - but the group began in April 2002… (A LOT of messages to not be backed up!) I can see them listed in the calendar, but they're not available to be seen, much less backed up. So if anyone else encounters this, it's possible that the earliest messages cannot be accessed anywhere at all for whatever reason, and thereby will not be able to be backed up unless Yahoo figures out what's going on with them (which I highly doubt they care about doing).

[doranwen]

Do you know if there's anywhere to call for people to see if they're a member of restricted groups with inactive moderators? I've applied to join several, just in case, but chances are good that the mods don't even check that email anymore and will never approve the membership. However, some of those have several hundred members and there's a chance that one of them might see the notice if there were a central place to link to with a post that people could send out, kind of "if you are a member of one of the following groups, please contact us right away to help with archival!" that sort of thing. I can handle joining any public group and grabbing all the messages (I haven't even been following the message restrictions recommended by the program, but haven't had a single lockout from Yahoo so far despite grabbing two 50+k archives straight…), but the private ones where the mods have abandoned them, those seem to be lost causes. :(

[capsulecorp_tm]

Thank you for ALL of this, I have a few mailing lists I need to archive for business and tax purposes more than fandom but...eh, it all applies. Bookmarking this post.

[duskpeterson]

I don't have time this weekend to read through the whole thing, but those are very helpful instructions.

[duskpeterson]

Thanks! I just read through your Google doc, which, along with the Open Doors post, makes everything crystal clear.

Thank you for your post to BIC, which I moderate. :) I belong to a bunch of other groups from the early 00s; I'll see whether anyone - such as you! - has posted an announcement; if not, I'll post mine. And I'll try to preserve what groups aren't already being preserved.

It's wonderful having you as a guide to all this.

[duskpeterson]

Morgan, on the doc page "Yahoo Groups Being Archived," it says: "To see a list of Groups being archived go to: https://forms.gle/4dvTgq5LFuGjrHKj8" But that URL just leads me back to the form where you add groups that are being archived. Is there a list yet?

[amedia]

GOT 'EM!!! And several other Ratty groups I belonged to, too.

Thank you so much for finding and disseminating the program throughout fandom. I'm sure a LOT of important history is being saved as a result. ♥

I watched the video and found the program super easy to use, so I haven't referred back to the instructions much yet. I was glad they were there, though!

I will probably use the instructions to export the messages, if I need help.

[morgandawn]

fixed. the url is

https://tinyurl.com/yyncxewj

[zippitgood]

Hi, I'm downloading several groups I'm a member of right now and had a question. Is the payment for the program a one time payment or is it a subscription?

[morgandawn]

Wilson the developer said it is a one time payment including all future upgrades. But you can ask him on his his forum (link is on the PG Offline Tutorial Guide)

[morgandawn]

What I can do is put together a Google form where people can submit the mailing list and it shows up on a spreadsheet (like this one: https://tinyurl.com/yyncxewj )

It will then be up to fandom to start pushing out the message