morgandawn | Yahoo Groups Shutting Down

Crossposts: https://morgandawn.livejournal.com/1715001.html

Entry tags:

fandom history

Yahoo Groups Shutting Down - Dec 14, 2019 (UPDATED)

(Updated Oct 25, 2019)

Starting Oct 28, 2019 no new files (photos, documents, etc) will be allowed to be uploaded. Starting Dec 14, 2019, Yahoo Groups will be limited to emails only - functions like digest will be eliminated and all files and photos will be deleted. Admins will have limited tools and all groups will be invite only. And archives of existing messages will not longer be accessible.

What To Do?

I have a step by step guide here. The short version

1. Admins - back up your Groups and decide whether and where to move
2. Members - back up your Groups - don't leave this to the admins.
3. Everyone - consider documenting Yahoo Groups on Fanlore, help spread the word

The OTW has made an announcement. Their Open Doors preservation program is open to providing storage of Yahoo Groups backups that are assembled by moderators and non-moderators alike. Details here.

Fans are gathering on a Discord Channel to chat and ask for and give help: https://discord.gg/ek2xFyJ (updated invite)

Please follow the Yahoo Group Rescue Tumblr for updates

Threaded | Top-Level Comments Only

yes, I spot check the year the numbers of messages balloon. Then go to most recent messages to see if the moderators or anyone else is active. Then cut off downloads in the year the bots took over.

I download all the files and photos making certain my virus filters are running.

I also have all of Henneth_Annun's messages backed up that were possible; sadly, numbers 1-16891 were inaccessible by all (no one can even see them on the site), but one of the members has several thousand early emails and is hopefully working on retrieving those in a form that can be archived. If you manage to get any of the early ones in some fashion, let me know! The numbers can be seen on the calendar, but clicking on them only brings up the February 12, 2003 (and after) messages rather than anything before that.

EDIT: I've also thought we need a list of locked groups - and not just of Tolkien, too. I've run into 10 so far that I can't get into and very much would like to, and they range from some Tolkien (though not on your lists anywhere!) to Pretender to L&O:SVU to Smallville to multifannish, for instance. Currently I have them open as tabs because I need to find a place to put them where they can be promoted. Perhaps a Google Form feeding into a spreadsheet as well? (And within the spreadsheet a link to "if you're a member, here's what you need to do!") I don't have the skills or time to create that, nor the social media accounts to spread it on, but if someone does create it, I'll put it out in the few places I can, and add the groups I know need backing up.

Edited 2019-10-22 05:17 (UTC)

We're working on a Google Form where people can suggest Groups to be archived (and mark them as 'locked' or 'private' to show that someone needs to become a member). It will feed into a spreadsheet like this one https://docs.google.com/spreadsheets/d/1TDxsfAl1lf20habGWyF_Sp-vIpNWnMyLhbXKcQYTQPo/edit#gid=0

It will be up to fans to find people who are members of the Group. You can also try sending a message to the list owner by NAMEOFGROUP-owner@yahoogroups.com - it is the same email that is used when you subscribe but maybe a different subject line?

Awesome! I'll be eager to hear when it's completed. :)

I assumed that they got an email at that address already when I applied, but it can't hurt. I've written up a nice template to use in emailing, and hope it will help with some. I've had at least two or three groups approve my membership (and one denied, but I could read the messages just by applying, so was able to back it up anyway), but the majority have been silent. Fingers crossed that this will elicit a few positive responses!

feel free to share the template

I think I am getting around 20-30% response rate for the 'requires mod approval to join'

Well, this is what I've been sending (replacing the ***** with the group's name):

Hi, I'm Doranwen, and I'm one of the many volunteers who are attempting to make backups of Yahoo Groups before all of the data disappears thanks to Yahoo's recent decision to scale back Yahoo Groups to mailing lists only. I have applied to join your group ****** in an effort to preserve this data. Let me explain what I will do, if you'll approve my membership:

- I will use a software program to download all of the files and photos for safekeeping.

- I will use the same software program to crawl all of the messages and save them to a database format (db3).

- I will export the messages to a single HTML file. (Future export possibilities include a nicer formatted HTML, as well as a couple SQL types.)

- I will upload these to a *private* Mega.nz folder, and provide you with the link to your group's messages and other files. I will also send a copy to the Open Doors project for preservation. They are committed to preserving the data, but will not share it publicly unless given permission by group moderators (indeed, at this time they have no framework in place for sharing it at all, but simply wish to preserve while we still have the opportunity to do so). You can see more information about these efforts here: https://archiveofourown.org/admin_posts/14117

Thank you for considering this request.

Doranwen

I've tweaked a few things along the way (just edited now to improve - I'm terrible at email, I always say more than I need to and have to keep paring down to the essentials), and I'm sure a few more things could be tweaked to improve it (for instance, maybe I don't need to mention the possibility of the nicer-formatted HTML files as I don't know if I can scrounge up $25 anyway - I literally don't even have that much extra this month, period, and it's hard to justify for that one simple purpose, as useful as it will be). But that's the gist of it.

I linked them to the post on AO3 because it's the most thorough explanation of the situation (and it links here anyway). And others may have a different method of sharing the files, but I think a Mega folder is probably the best option, at least for me. It's secure (I can provide only the link to their data and not the entire folder - and no one else can access anyone else's with it), permanent (no 30-day expiration after the last download or something similar), and will hold a whole lot of data.

Edited 2019-10-22 07:21 (UTC)

Which groups am I missing? I've mostly just been putting LOTR keywords into Yahoo Groups Search or the list provided by the Archiveteam. This has limited me to searchable groups.

If you tweak your e-mail a little to address it to general readers/any mod who happens to see it, I can post it at tumblr to see if it gets some traction and catches any former mods' attention.

Here is a Discord invite to the Save Yahoo Groups Server:

https://discord.gg/qbMJJp

Here is a link that explains my way of saving:

https://1drv.ms/u/s!AttWx8rXBtV-gsATN7wocVi4UG1d4g?e=ulpaWt

Here is a Discord invite to the Save Yahoo Groups Server:

https://discord.kolob/qbMJJp

Here is a link that explains my way of saving:

https://1drv.ms/u/s!AttWx8rXBtV-gsATN7wocVi4UG1d4g?e=ulpaWt

The json are not very readable in their current form. To do so, I've been using a text editor (Textpad) with regular expression support and replacing "summary" in the files with \n"summary" to make new lines where the messages start.

Many, many thanks for this pointer; that may just solve the problem for the one good-sized and still active list I'm on. (Someone else has already grabbed an archive of the content, but we have been dithering about where to migrate, and the group in question is one where there is both fanfic and author-event content.)

I meant from your "locked out" list, there are some LOTR groups that I have attempted to join and gotten nothing back from the mods, yea or nay. Perhaps that's not what you meant! These are three that I've spotted so far: aragorn-legolas, fanfic_of_arda, lordoftheringstrilogy
The first is particularly large, tons of messages - but you can't see them unless approved as a member, alas! (Some say you need approval but as soon as you apply you can read the messages, so while I can't get files/photos from them, the messages at least can be backed up.)

I'm still working my way through every LOTR one I can find, and then I'll probably turn to X-Men. I've already gone through all the Pretender ones and all those for smaller fandoms that I could think of.

I'm not sure what you mean by "address to general readers"? Like, do you want more of an open letter of the "are you a moderator/owner of a Yahoo group? consider letting one of the following volunteers back up your group's data for you" kind of thing? Or what? Sorry, just need a little more clarification there.

I can't handle Discord, because the emojis bother me a LOT. I find them extremely irritating to the eye and sometimes even confusing. (There's one on Slack, for instance, that's supposed to be "confused" - it doesn't look remotely confused to me. If I went off the visual, I'd use it for "sympathy"!) So I turn off emojis on Slack and I loved using IRC when the Yuletide channel was active there. Discord, sadly, refuses to even consider allowing users to individually control their viewing of emojis whatsoever, and when I tried suggesting it on their forum I had dozens of people downvoting it (apparently people get very angry when you want to NOT see their emojis :P so frustrating). Discord is clearly not neurodivergent-friendly (as I'm pretty sure my Asperger's has something to do with my strong dislike of emojis), and I wish there was a way to change that, but no one's listened to me there! So I don't use it at all.

I run Linux on my usual system and fired up a Windows 7 box I rarely use just for this purpose. Perhaps ironically, I actually don't know scripting; I can do some cli stuff but prefer a gui whenever possible, hence using PG Offline. (I do know a little SQL though, so I'm hopeful that I'll be able to figure out how to export stuff using SQLiteStudio even if I can't afford the cost of the registration for PG Offline).

I hope it works out well for you. :)

I have deleted the previous messages because they contained inaccurate information. WARNING! This JSON (https://groups.yahoo.com/api/v1/groups/concatenative/messages?count=100) does not contain the entire message past a certain length. Longer messages are cut off!

Here is an updated version:

There is an API provided here. There's also a list of Groups-scraping software on the page:

https://www.archiveteam.org/index.php?title=Yahoo!_Groups
https://groups.yahoo.com/api/v1/groups/concatenative/messages/1/raw

If you use this API, it tells you how many posts are in the community after "totalRecords":

https://groups.yahoo.com/api/v1/groups/concatenative/messages?count=1

From the Archiveteam, this is a list of the names of many public Yahoo Groups. But not anywhere near all of them:

https://1drv.ms/u/s!AttWx8rXBtV-gsAD1rdmzrGDuiIE6g?e=tYQNcF

From the Archiveteam, this is a list of the metadata of many public Yahoo Groups. But not anywhere near all of them (14GB file):

https://1drv.ms/u/s!AttWx8rXBtV-gsAExsAo15VXYELjUw?e=WJtEoa

Here are some good lists of fandom Yahoo Groups though many on them are deleted now:
http://paula3001.bravepages.com/IRFTIA-mailinglists.html

There should be multiple back-ups of the important communities, so I don't want to stop any of you but I'm in process saving 638 Tolkien-related groups using Autoit's Inetget.

https://www.autoitscript.com/autoit3/docs/functions/InetGet.htm

https://1drv.ms/t/s!AttWx8rXBtV-gsAKwAXENEIQW3D6BQ?e=jDGubc

But this is a list of groups I am locked out of. I have applied to join them but have gotten no response because they are possibly abandoned:

https://1drv.ms/t/s!AttWx8rXBtV-gsAFyubXjC6n_Ux1-Q?e=4plgbh

Any info on any moderator of any group on that list would be good. We should try to spread a list of locked groups around Tumblr with a project name like Unlocking Yahoo or something to try to reach moderators who have abandoned their groups. Or at least members of these groups who could save things on our behalf. Do you think the OTW would post it to give it some traction?

Yes. What you said is what I meant. I want to send something around tumblr that will perhaps alert any mods who see it.

I understand about Discord. I like emojis. Though not so much that I want to see them EVERYWHERE. I've liked them less since they've become commercialized.

I just realized the very simple solution... copy the database as many times as there are groups, rename each one to the group name desired in it, and then go in and delete the ones that will be kept in other groups. Can't believe it didn't occur to me till now!

*waves hi as the "someone else"* I didn't greet you directly on there but I do recognize your name from Yuletide and other places, and glad we can work together to preserve the LOM data. :) I'm waiting till the last posts have gone on the site and been frozen (next Monday) and then will start emailing owners with links to where they can download their data.

I've got something like that started, but realized I had a lot of questions about what it should contain.

For instance, do we just link them to the OTW post? Or should we have something like this:

- If you want to preserve the data yourself, go here: *link to instructions on software to do that*

- If you'd like someone else to preserve the data for you, go here: *link to... something?* (don't currently have anywhere to say "go here and ask for someone to join your group so they can back things up, but it's more likely that they've got multiple people asking for membership to do just that - and if they're touchy about the data getting all over the 'net, how do they know that the individual person is going to be honest about it? I mean, if I tell them "I will not post it publicly", I mean it - I never lie about what I will or won't do - but others may not be as honest)

Also, I'd think they ought to be linked to the list of "we need these groups backed up" - and if that is the case, should the message go out to all members of locked groups instead? Because as long as someone's a member, they can do the backup, even if they can't get mod permission to share it.

And should they be linked to the list of already-archived groups? So many questions I have. If I get some answers to those, I can work on crafting a simple, straightforward message. (Which I can do if I work at it, lol, I just have to fight my temptation to over-explain and over-share.)

It's all right. I was going to share your questions with the Discord but someone already wrote a public tumblr post:

https://yahoo-geddon.tumblr.com/post/188528746979/yahoo-groups-will-be-shutting-down-what-each-of

Hi,
Just a lurker, just wanted to point out that the extension also works on Windows if you tweak it. You have to repackage it into a crx file using the YahooApplication folder. Once somebody creates a crx file it can easily be shared. It's not a large file.

I'm actually running out of groups to backup. I've looked at everything I can find from search, and they're all one of the following:
- locked (and very few of those have allowed viewing of the archive upon applying - I'm not sure how the ones that did, did)
- nothing but spam (it's not worth it to me to set it all up to download just 15 posts of teens going "come join my group!" and then stop because the rest is spammage)
- only for certain types of content that I'm not into
- fandoms I'm not into (I'm happy if someone else backs them up, they just have to be interesting to me for me to put the effort into doing it myself).

I've added all the ones I can't get into (that haven't outright denied my application to join, as a couple have) to the spreadsheet as suggestions (and am delighted to see that one's already been claimed and backed up).

I've tried every combination of words for the following fandoms (I'm now resorting to slowly browsing through long lists in various categories in hopes of finding promising ones): Tolkien, Pretender, X-Men Movies, Smallville, Narnia, Dr. Quinn, Lois & Clark, Women's Murder Club… I even ran across a Black Sash group, which is so obscure it doesn't even exist on AO3, and I'm planning on requesting it this year for Yuletide!

Any suggestions for how to find more fandoms to back up, without spending hours (that I don't really have) browsing through the directory of groups in an attempt to find good ones? As long as it's a fandom I'm remotely into (anything in my interest list on the profile) I'll happily turn my computer loose on it.

EDIT: I forgot Narnia…

Edited 2019-10-24 05:05 (UTC)

A few fans are going through old lists of Yahoo Groups looking for ones with automatic member approval a s should have a that list soon. We also have a few on our Google Spreadsheet...those have not been vetted yet. I'll try to post the info as soon as I get it

Awesome. :) I claimed one on the sheet that listed Pern (a book series I enjoy, if not as fanatically as some fandoms) as the fandom (though it appears less Pern than a mix of fandoms, and mostly ships I'm very much not into, but hey, it's backed up now).

I'm slowly poking through the lists of groups under All / Entertainment & Arts / Humanities / Books and Writing / Fan Fiction - found one or two potentials so far.

Here is one on our spreadsheet no one has claimed: https://groups.yahoo.com/neo/groups/MSRfanfic-only/info

X-Files is one of those fandoms I'm definitely not into and never will be. (The only alien I like is Clark Kent, pretty much, and the little bit I saw of X-Files once creeped me out.) If it's still lingering when I've completely run out of fandoms, maybe I'll claim it so it's not lost entirely, but I'd hope someone else would get it that would actually be interested in the content in addition to preserving it for preservation's sake.

Anyway, earlier I stumbled across a fanfic resource group for medical details and decided I'd hop on and snag all of that (useful stuff), and there was a LOT (they were smart enough to require approval of messages from new members, so no spam to worry about); it's still plowing through that one and will probably do so for another few hours, by which time I'll be asleep.

Threaded | Top-Level Comments Only

Page 5 of 6

Yahoo Groups Shutting Down - Dec 14, 2019 (UPDATED)

Re: A helpful tool

Re: The Great Purge

Re: The Great Purge

Re: The Great Purge

Re: The Great Purge

Re: The Great Purge

Re: The Great Purge

Re: The Great Purge

Re: The Great Purge

no subject

Re: The Great Purge

Re: The Great Purge

no subject

Re: The Great Purge

Re: The Great Purge

no subject

no subject

Re: The Great Purge

The Great Purge

Chrome Extension

no subject

no subject

no subject

no subject

no subject