Blogger

February 25, 2001 10:16 AM

Blogger collapses once more - this time with the message: "Error 103:java.sql.SQLException: java.sql.SQLException: The log file for database 'pyra' is full. Back up the transaction log for the database to free up some log space. [more info]" It's an old theme, but a pertinent one to a large number of people on this site so don't kill me. It's being talked about here. Look - we all know that this is not the fault of Pyra, but it's getting absurd. Prol has already migrated over to Grey Matter - and from the look of the mailing lists, she may be the beginning of a flood. I have so much invested in Blogger working, that I'm prepared to be slapped for saying something must be done. What can we do?!
posted by barbelith (37 comments total)

As a Blogger user who is contemplating a move, I would like for a subscription-only server to be set up. Battling for resources with all the other free Blogger users is tough.

Anyone who is comfortable with XML can easily migrate their weblog out of Blogger by switching from HTML to XML templates and republishing the whole site.
posted by rcade at 10:24 AM on February 25, 2001

Something *must* be done; what I don't understand is why Blogger can't get millions and millions in VC funding.

Okay, so there's a mini-recession on and the technology industry has been particularly hard hit, but Blogger's so obviously one of those few 'killer apps'.

Surely potential investors can see that Blogger has become a true phenomenon of the Web and is quickly becoming a valuable brand.
posted by tobyslater at 10:24 AM on February 25, 2001

Tom's right, I definetly see a lot of people moving over to custom scripts in the future, being that it gives them more control over their content. I've had a number of emails already, asking for my community scripts and personal scripts.

If things like this keep happening will it filter out the noise? Will everyone and their grandmother no longer have a weblog? Is that a good or bad thing? I'm not sure yet.
posted by tomorama at 10:32 AM on February 25, 2001

A few things.

- No offense Tom, but I'm tired of seeing "blogger down?" threads here at metafilter. Threads like this are meaningless to everyone except those that use blogger, and even then, users know to expect it. I know your post was more than that, so I let it stay. I'm tired of seeing all the "is this site gone?" threads though. MetaFilter is not everyone's uptime service.

- the database transaction log error comes up every couple months, isn't something that can be predicted, and yeah, no one can post until someone logs into the servers and deletes the old transaction logs. It's nothing new, but now that there's only one person running the show, the chances of them finding out and fixing it are slimmer than when there were five there.

- Everyone may be running to greymatter. So? It's a great tool that does lots of things blogger does. It's hard to setup though, so no matter how many people start using it, it's most likely just the power users fed up with blogger. I've used it, and recommended it to many, but only to the ones that know what unix filesystem rights are.

- "we all know that this is not the fault of Pyra, but it's getting absurd." Who else's fault could any of this be? (I put myself in there as to blame, since when pyra was really chugging along, we could have spent more time dealing with error checking)

- "What can we do?!" You can build your own weblog tool, you can migrate to greymatter or manila, or you can send Ev a large check so he can hire some people to fix the bugs.

- I don't think VC funding is going to fall from the sky because there's no big profit source for Blogger. If you were a VC guy with millions, and you gave it to Blogger, the most you could hope for would be millions of people using the product and paying a few bucks each. VCs don't want to make a few bucks per user, they want a lot more. They like to see thousands of installs of $100k enterprise versions, they don't like consumer sales.
posted by mathowie at 10:36 AM on February 25, 2001

i have to second matt's comment about Greymatter. i use it to manage our weblog and have been very happy with it. but (and noah is very upfront about this) it's not for everyone; there are hundreds of template variables involved, and it takes time to puzzle it all out.
posted by cfj at 11:02 AM on February 25, 2001

until noah fixes the performance issues, i cannot imagine any power-user migrating to greymatter. when i say "migrate" ... i really mean migrate existing blogger-powered content to greymatter. when i evaluated gm, i "exported" a little over a year's worth of content (~1200 posts) to gm format and after over 2-hours of grinding, gm still was not half-way through the rebuild process.

overall, i am very impressed with the ui and functionality, but the issue i described above, for me, would be a show-stopper.
posted by chrish at 11:29 AM on February 25, 2001

Just to add: the performance issue chrish mentioned involves any files you optionally choose to connect to Greymatter, not Greymatter's own entries - when you add a new entry, GM only rebuilds 2-4 relevant files, not everything (so GM doesn't really get slower as time goes on). I would think 99.9% of people would never have a need to externally connect hundreds of files (or even more than a dozen at the most) to GM - most everyone that migrates to it from Blogger just links to their old archives, since there isn't a Blogger-to-Greymatter conversion tool (yet). But I've worked with plenty of sites whose GM entries now number well into the hundreds, and the performance is just fine; even a top-to-bottom rebuild on most sites with hundreds of Greymatter entries is usually just 5-10 minutes.

and no, it's not meant for everybody; I can't say enough that anyone looking for nothing more than a mere alternative/replacement for Blogger is likely to be disappointed. Most who are very happy with the way Blogger does things would, all in all, probably be happier sticking to it and waiting for it to sort its glitches out.

(By the way, I wonder how common it is to experience a sinking feeling of dread when you check your logs and find hundreds of visits from metafilter.)
posted by Noah at 11:52 AM on February 25, 2001

(More common than you might think, Noah.)

I'd like to second Matt's thoughts. As a MF user, I'm getting tired of the "is this site down" posts. Nine times out of ten, the site is up and chugging along just fine by the time I get there. Besides, it just doesn't make a very interesting thread.

As for the eternal what-can-we-do-to-save-blogger thread, if we/they couldn't figure it out in all that time, I doubt we will here. I'd prefer to just let this one alone and hope for the best.
posted by fraying at 12:00 PM on February 25, 2001

Matt's right, VC funding doesn't fall from the sky. You have to ask for it. You have to do a lot of work predicting why your business is going to generate X return on the VC's investment, you need to make a presentation, you need to spend a lot of time on it. Also, you have to *want* it. And I don't believe any of that's happening at Pyra right now.

I'm in agreement with Derek on this one: MeFi users aren't going to come up with a way to save Blogger. Every idea that's ever been posted here was already discussed by the company long before. There's never been a shortage of ideas on how to generate revenue for Blogger. There's just been a shortage of execution.

As for "what can we do?" Well, not a whole lot. Stop using it. Wait patiently until the situation improves. Try to engage Ev in a dialogue about what future plans are for things like a subscription-based Blogger with a dedicated server. Organize people to volunteer their services, or volunteer your own.

I don't mean to belittle the point, and I understand the current frustration. But really, there's just not a whole lot that can be done. There's one person running the show at Pyra.
posted by megnut at 12:23 PM on February 25, 2001

Matt, this *is* a weblog, albeit a collaborative one. And you use Blogger yourself, so there is relevance.

If you don't want any further posts regarding Blogger's downtime, you should either remove them or add that specific exclusion to the posting instructions.

I have found this thread to be extremely helpful, as it happens.
posted by tobyslater at 12:53 PM on February 25, 2001

while i agree that "is this site down?" threads can be boring and non-productive, i think the case of blogger is unique.

it is a highly valuable, highly respected and widely used tool. it is representative of what is wrong with the industry today -- a good idea not getting the support it needs because of all the bad ideas that blew up in the past.

i feel that discussion is important, whether in a formal setting or not.

if we (not 'we' at MeFi, 'We' in general) hapen to find a solution to the blogger issue before Ev does, it is likely to be grass-roots-based, and may even be a model solution for other resources that we want to keep, but whose keepers can't afford to (like MeFi itself in the future?).

i also think (to take a contrary position to my first sentence) that we do too much harping about 'non-relevant' or 'useless' links or threads. it is relevant to someone or it wouldn't have been posted. and the purpose of a community blog is not to entertain all of the members all of the time.

if i agree to stop posting links about blogger and the demise of other popular sites, will you agree not to post any further physics- or astronomy-related links? 'cause i really don't care for them...
posted by o2b at 1:21 PM on February 25, 2001

Off-topic: Does anyone know the specs of the new Blogger server?
posted by hobbes at 1:57 PM on February 25, 2001

If you don't want any further posts regarding Blogger's downtime, you should either remove them or add that specific exclusion to the posting instructions.

I have been removing them, as I have removed several others about sites being down (word.com being the most recent one). I realized this morning that a thread taken down wouldn't help, because people wouldn't see it, and wouldn't know how exhausting it can be to read threads that question uptime of a site. So I left this one here, with a warning, which if ignored, will eventually make it into the guidelines.
posted by mathowie at 2:21 PM on February 25, 2001

May I humbly again suggest that if we must discuss Blogger, Greymatter, weblogs down and whatnot that the "Weblogs" category over in MetaTalk, right here in River City, might be a more appropriate place?
posted by bradlands at 2:38 PM on February 25, 2001

Ev's fixed it now, regardless. And I second Brad's suggestion for future threads.
posted by anildash at 3:54 PM on February 25, 2001

I understand completely the reaction of people to my posting today - these threads are fairly common, it is true. I'd link to a news story that read: 'power blogger users may be migrating to other systems' instead if I could. My issue here is not so much with the tool or the company, but with the community and the preservation of it. No, I do NOT agree that Pyra is to blame for the performance issues associated with Blogger. Anymore than I think a charity or community dance project would be to blame if too many people put demands on their service. I wanted to take the conversation away from the mechanics of 'Blogger don't work nice' to a new conversation, the simple question - as a COMMUNITY rather than as 'consumers of a company's services' - is there anything that can be done to help - suggestions running from appealling to people to use Blogger less, or to restrict their usage to off-peak times, right through to physical volunteering to help. And I also wanted to discuss with people what possible consequences the current impact of the state of play at Blogger might have on the development of that community etc.

I just want to repeat that I am aware that the initial post wanders over the line into repetitive posting on occasion and that the appropriateness of it is suspect. But I also stand by the assertion that while this isn't a blogger discussion forum, nor a place to bemoan up and down times, it is still fundamentally a collaborative weblog, with a substantial readership of people who both produce weblogs and read other weblogs - a readership (either way) to whom the consequences of major Blogger problems, and the state of the weblog nation are valid concerns - of interest and are worthy of discussion. Perhaps not in this case, of course, I might have over-stepped the mark - but in general, news about Blogger seems to me to be news about the progression and development of this medium and to an extent independent content publication on the web and hence worthy of a place. I will of course bow to public opinion on this one, and if people consider me hopelessly naive or clumsy then I'll bow to their judgement.

Finally, sorry if I've pissed anyone off - that was never my intention...
posted by barbelith at 4:17 PM on February 25, 2001

"What can we do?"

How about getting a real life outside of the computer world?

it's a start....
posted by Capn_Stuby at 4:59 PM on February 25, 2001

Now, I hate to have to be the one that mentions this, but when I read the tone of this thread and statements like "wait until the situation improves," I have to ask: Has anyone noticed that is *has* improved? Even a little?

I understand it's human nature or something to notice when things don't work and not when they do. But, I've been working hard to improve things, and I feel I've made a lot of progress! According to my logs, 18 days ago, I made some server upgrades that dramatically improved things. Since then, as far as I know, the speed of Blogger has not been a problem for anyone (Blogger Pro beta users excepted). This was for the first time in months -- even though we got 20K new registered users last month alone.

And since that time, there have been a total of two significant outages. The first one, also well documented on MeFi, thanks, affected only half the users, who couldn't get on one morning. Today's affected everyone, for about 15 hours, because it happened to occur during the one significant chunk of time during the week when I was actually offline.

Now, two outages in 18 days is way above an acceptable level. But it's better than we were doing before! I'm sure y'all will have trouble believing me, but I really need to stress that things are not getting worse. They are definitely getting better -- even though it is one person here. In fact, they are partially getting better because, even when we had a lot more people, we were very distracted and stressed out by other things, like payroll, which keeping the servers up didn't immediately contribute to. Now, maintaining and improving Blogger is my focus. I'm putting backup systems in place, improving the architecture, and making deals that will provide the revenue. Things are looking up.

As for VC money, Meg is right: I have no interest. VC's don't give a shit about personal publishing. And I have no interest in making Blogger into, primarily, an enterprise software tool or selling to CIOs. I want to help people express themselves, build communities, and create the future of journalism.

If Graymatter is for you, go for it. If you do truly want to help Blogger, mention when it's working well, not just when it's working poorly. :)

Peace,
Ev.
posted by evhead at 5:14 PM on February 25, 2001

At which point I want to withdraw my initial comments completely and unreservedly and humbly ask that the thread be removed.
posted by barbelith at 5:26 PM on February 25, 2001

Which is another way of saying that I feel like a complete prat for writing something which doesn't adequately express my feelings on the matter clearly, and which has left me in the position of trying to explicate rather unformed impressions not all of which I have particular faith in. Apologies all round, but particularly to Ev.
posted by barbelith at 5:29 PM on February 25, 2001

Actually, now that you cleared things up Tom, I think it was a valid question. What can we do? Anything? Nothing?

(is it ok if I leave this up?)
posted by mathowie at 6:26 PM on February 25, 2001

i, for one, would love to know what i could do.
posted by o2b at 6:39 PM on February 25, 2001

At which point I fear it may be more useful for me to re-self-blog the Blogger Pro info page I made a while back. While it hasn't been updated in a while, it should help the conversation.

And the page has a poll on how much you'd pay for Pro. Should be of interest...
posted by anildash at 9:56 PM on February 25, 2001

i noted with interest that someone mentioned above a subscription model, but with not much followup.

i second that motion; there is absolutely no reason pyra should be expected to run this as a free service forever.

meg mentioned these ideas have been gone over before...why was the subscription model chucked? (just curious.)
posted by patricking at 11:32 PM on February 25, 2001

there is absolutely no reason pyra should be expected to run this as a free service forever

Except that's how Ev (who now is Pyra) wants to run it. I say kudos to him for wanting to keep it the way it is, and he's right about Blogger getting better. Except for the error last night/this morning it's been working just fine for a few friends and I who have a blog together. Although I think it would do Blogger good to look into using the Amazon honor system and perhaps a P.O. Box, if nothing else, because there are definitely people who would like to give small donations to Ev and Blogger just as they would like to give, and have given, donations to Matt and MetaFilter, not to mention Gabe and Tycho over at Penny Arcade.

Is the paypal link that was present during the server crisis still up? If you wanna help out Blogger and you don't live in SF I would think that would be a good way to do it.
posted by crushed at 3:44 AM on February 26, 2001

Sorry, I guess technically it was night before last/yesterday morning. Please excuse my bad sleeping habits. :)
posted by crushed at 3:46 AM on February 26, 2001

The thread's still here, so I just want to reiterate that I wasn't so much talking about community stuff in terms of money, but instead best practice for webloggers / volunteer work etc. Would Ev be willing for people to develop additional features for Blogger - is there a possible distributed approach to dealing with the ever growing databases that might work etc. etc. Let's be more creative here than just (although I agree with the Honor System thing) throwing money at them...
posted by barbelith at 3:54 AM on February 26, 2001

For what it's worth, I'd rather see a million "death of Blogger?" posts than one more "All Your Base" post. But yeah, there's a whole section in MetaTalk for something like this.


posted by thc at 4:54 AM on February 26, 2001

I agree that there should be ways for the Blogger community to help Ev out when he needs it. However, since not everyone who uses Blogger has the skills that might be neccessary, the honor system or something related may be the only way for some people who want to help. That's why I mentioned it. The reason I said people who don't live in SF was because I seem to recall Ev needing a person or two to help him out with stuffing enevelopes one night. Volunteers such as that I'm sure would be a pretty good help to both Ev and the overall application.
posted by crushed at 4:54 AM on February 26, 2001

Well, Blogger isn't open source for a good reason. Coordinating a myriad of developers, getting code to them and checking code coming from them, incorporating it into the existing codebase, testing code, and dealing with their wishes and wants vs. Pyra's wishes and wants is a huge undertaking.

I spoke with a person in charge of some open source projects who admitted that most times, it's more trouble than it's worth, and the output you get from random coders isn't as good as employees. Also, it may lead to some sort of insurrection when your volunteering coders want to build something into the product that the company doesn't want.

Then there's the whole sticky issue of volunteering for a profitable company, and how reward mechanisms work out, if at all.
posted by mathowie at 8:49 AM on February 26, 2001

Matt, don't you mean a "for-profit" company? I don't know if we're ready yet to call Pyra a "profitable company." ;)
posted by megnut at 10:44 AM on February 26, 2001

"profitable" works if Matt's a forward-thinker. :-) It may not be a problem figuring out rewards for volunteers of a company that isn't profitable - "Here, you each get 20% of nothing! Bwahahaha!" - but it's when profit starts coming in that causes the problem. :-)

I'm going to go off a smidge wildly. This isn't a real (or plausible) suggestion for Ev, it's more of a vocal thought excercise.

I think it'd be possible to open the code base completely and still have Pyra the company be a discrete entity. If the code base itself is free, then the only barrier is servers and bandwidth, which is a pretty big barrier. A company (preferably a profitable one though :-) could serve up the Blogger utilities. Don't quite see how that'd make for a profitable company though, because the real power in Blogger's the code.
posted by cCranium at 12:04 PM on February 26, 2001

Hmmm, I might be splitting hairs here, but IMHO the real power in Blogger's the interface. I would guess that Ev and the folks who used to be at Pyra would agree that a form with a text area to enter stuff to be FTPed isn't technically very challenging.

Truly, it's the stellar implementation of it that makes Blogger rock as thoroughly as it does.

And as for making money off the Blogger code and interface, I'd say there are lots of ways to do it that compromise what Blogger is, unless it's used kind of like Trellix is, and then licensed to a Geocities or MSN Communities or something...
posted by anildash at 3:21 PM on February 26, 2001

The profit thing I completely get - the whole recent appeal for money made me consider which other companies I would simply donate money to just because I liked them. Answer: probably none, which makes me think that what we are talking about is a community saving procedure rather than a company one.

In fact, quite a lot of the blogger community was formed directly through the now absent search pages - being able to search for other people's replies to your writing was always a tremendously stimulating and rewarding process. Perhaps a third party could generate an external database designed specifically to track loosely bracketted time-specific changes to weblogs - external to Blogger (or indeed any other species of weblog) - and generate a search engine from that.

Just out of the blue in my head: What's the position on micro-bloggers? I mean - would it be possible to distribute the blogger databases over more than one location / site? Don't know what the advantages would be. Would there be any?
posted by barbelith at 4:57 PM on February 26, 2001

anil, that's where I disagree. Well, that's where I get out my laser and start splitting hairs even finer.

The interface is part of the code base, and yes, it's very important, but I've got the sneaking suspicion that there's a whole lot more power to the backend than is available to most Blogger users.

And I don't mean that Pyra's holding out, and I'm not even just referring to the various Blogger Pro enhancements. Blogger, as we all know, started as a bit of a toss-off from Pyra The App, which from what I've seen indicated was a massive groupware application. (I think I'm genericising it a bit).

You can see hints and peaks of what's possible with the way Blogger's implemented. Things like the Team Management area indicate to me a back-end that's been designed to allow for reasonably painless integration of new features.

Again, I'm in the realm of pure speculation here, but Blogger has a shit-load of power behind that's just waiting to be exposed. Unfortunately, I think that the exposing of it is an expensive, time-consuming task, and because it's capable of so much, it's hard to focus on an area and get that out.

barbelith: considering the relative ease in which Pyra tossed in the new server, I imagine that distributing it widly would be technically feasible. Plus it's running on mostly NT machines, and while the advantages are debatable, one thing MS does well is server farms.

It's an inherent part of their architecture, a server farm for the actual database, a server farm for the data transformation layer (dlls, or from what I understand in the case of Blogger, Java Servlets) and a server farm for the presentation (the web servers).

Putting the individual boxes in geographically diverse locations is definitely possible (especially with a Win2k VPN) but the communications time can take a hit, and when you're the size Blogger is, 4 or 5 boxes in one rack somewhere are generally better than 4 or 5 across the nation.

The obvious advantage to multiple servers is you get access to more processing power. More boxes to handle the database requests means more requests being handled at any given time. The advantage to multiple locations is reliability - if, oh, say, San Francisco suffers from some kind of power outage :-) then your New York boxes can take over the load.

Another disadvantage would be the cost of mirroring all the data. Every computer needs to have identical information, which adds a non-trivial amount of overhead to the mix, especially if the data has to be mirrored over any distance.

Yipes, didn't mean to ramble that much. It's one of those thinkin' days.
posted by cCranium at 6:03 AM on February 27, 2001

The slowness caused by enormous simultaneous traffic will ebb at some point. I don't expect huge tides of new users every month to go on forever. In fact, I'm hoping that a majority of these new users who jump on the bandwagon because it's cool to have a blog will fall off just as quickly. Resources to those who truly have a passion for personal publishing, not just some fleeting fad-like interest.
posted by susanlucci at 8:39 AM on February 27, 2001