Back now? Ok, Daylife (in case you’re cheating and reading ahead) is a “news aggregator” — that means it is a service that, in their own words, “gathers and analyzes mind-boggling amounts of high-quality news and other content from across the web.” Then they package it up in ways that virtually anyone from users (that would be you when I told you to go look at their site) to web-head, code monkeys like me can use.

Now that right there is very cool in and of itself, but the truly over-the-top feature of Daylife is its ability to “chain” topics. Like Wikipedia, you can spend hours clicking through links on Daylife meandering from news topic to news topic (until, I imagine, you end up at Cargo Cults since every web search ends there).

But so what? Well, I got to start looking at Daylife for work. Now this is really interesting, because when I first heard of Daylife (probably via BuzzMachine) it was billed as a way for small web news sites or beat blogs to rival their bigger brethren in content so right away it was strange — being the bigger brethren and all.

We decided (meaning, “I was told to”) to look at Daylife in three areas (and yes I’m being intentionally vague here because we’re not done yet):

1. Pad out an existing strong section with related content from Daylife
2. Bolster a weaker section of our content with related content from Daylife
3. Build out a section that we think could ‘drive traffic’ where we had no existing content before

It’s that last one that was the most interesting to me because, simply put, I thought it was terrible idea. Padding out native sections with related news is one thing, I think that’s a great approach (especially when you can tailor the sources like you can with the Daylife API) and as a believer in the Ethic of the Link it all fits in nicely.

(Hmm, all that Link Ethic stuff is from BuzzMachine who’s proprietor, Jeff Jarvis, is a partner in Daylife… do you see a pattern here?)

But building out what amounts to a virtual section of your web site with no original content at all just feels like, well… cheating. Sure your ad folks can probably sell it (revenue!!!), but it none of that content is yours and I question anyone’s ability to drive sustainable traffic to a section that’s not a natural strength of the site in the first place. And then I can’t get past the whole “negative-sum” notion of it — if everyone simply linked to everyone else, would anyone have any original content? To me this breaks the ethic of the link in a big way.

All that all being said, I still elected to start with this so-called “virtual section” because, ironically, due to it’s very nature of not having any in-house content it would be the quickest and easiest to build. This way, my thinking went, I could wade into the Daylife API, play around a bit, build the section out, deploy it and then move on to the “real” sections with a better understanding of how Daylife works. That seems to have been a good approach so far. I still think the whole virtual-section idea is dodgy though .

On the tech side, Daylife has a pretty slick RESTful API (called the DayPI… get it? day-pee-eye?) where you can do generic searches or pull related articles and photos for a given article or photo which is all to be expected. What’s cool is you can pull related topics as well, which come with their own stories and photos (this is the basis for the “chaining” mentioned above). They have a number of libraries and code snippets available (of varying quality) to get you going and I played with a few of them to get a feel for what was doable.

Ultimately, since they offer a native JSON return format, I opted instead to write a simple proxy to their API so I could roll the whole thing up in AJAX calls (made extra simple thru jQuery). This also allowed me to cache the requests on the local side for a quick speed boost.

So we’ll see how the various sections do, I like the idea of Daylife and I like their approach to news. If it’s used correctly it can be, I think, a really nice add-on for most news websites.

So I’m loving the new gig in the big paper’s newsroom. Despite the hype, I don’t think I’ve initiated any earth-shattering wizardry that will instantly “save the newspaper” from the perils that it faces.

Instead, what I have done is start with a series of baby steps, and mostly invisible ones at that. For one, I’ve been “fixing” the paper’s RSS feeds, starting with simply adding the reporter’s bylines to them. No, we didn’t have any bylines in our RSS feeds, go figure. Next up came photos: using Yahoo’s Media RSS extension I added each story’s photos to the feeds (yes, including the photoby ).

This, while useful in it’s own right (it looks very cool in Google Reader), it is a “baby step” on the way to other things. Now if I wanted to, I could, say, pull stories by reporter — or current photos by story — and place them on other sites. If I wanted to….

I’ve also gotten to play the roll of agent provocateur, a roll that gets little “press” but I don’t mind.

Watching our now photo-capable RSS feeds update throughout the day really illustrated something troubling: we’re really slow with updating photos. We can break a news story in seconds but for photos we usually have to wait for the shooter to come back to the building. It only made sense that I jumped at configuring a Nikon WT-2a for wirelessly sending photo’s off the camera.

While it was originally poopooed, when it was shown to work, it got some interest and now the paper has invested in another unit (a WT-4a for their newer D3 bodies) and when I started asking about Bluetooth GPS adapters for the photographers, I got some real traction instead of, “who are you again?” (geocoding photos and stories is another “on the list” thing of mine).

Also, it was I who passed along the specs of the Nokia N95 and info on Qik (not to mention helping to configure them) to the photographer who ultimately took them to the Olympic torch relay.

So I’m doing my thing… slowly, in baby steps.

That’s interesting, I thought, doing my best mental Jack Sparrow imitation.

You see, I don’t technically work for the Bee, but I often work at the Bee (ok, yes, I did once work for the Bee but now I don’t… it’s confusing) ANYWAY not long ago the AME in charge of the Bee’s “online operations” casually mentioned, “oh, hey, Rick wants to meet you.”

Say what?

Now, I know very well, I’m “known” around the papers I support. I tend to speak my mind and have these crazy web theories and opinions that sometimes impress people but sometimes…. um… not so much. I honestly asked, “uh, is this a good thing or a bad thing?” It was good thing, I was assured. “Maybe lunch… next week?”

That was two weeks ago.

I walked past the AME the other day and casually asked, “Yo, where’s my lunch?” with a grin. Apparently Rick had been out all week, and it was not normal. I don’t know newsroom politics really at all so I had no clue how to read that. Hell, I had no clue how to read any of this. Is it normal for exec editors of papers to, “want to meet you?” It had a weird Sopranos feel to it.

“…differences are not about resources, they’re not about staffing, they’re not about expenses…”

Well Jiminy, what does that leave? News coverage? Internet strategy? More importantly what about my lunch?!?

The next editor will not be a Bee employee but will come from the ranks of the newspaper’s owner…

Oh I can think of a few people offhand, it’s a zillion-billion to one shot but a certain Indians fan…

Welcome to my nightmare.

You’d think it would be easy, just issue a SELECT * FROM news WHERE date = $today; and be done with it, right? Well, not so much.

1980 called, they want their markup back

The first hurdle is that a fair percentage of newspaper publication systems are either old or based on old underpinnings. The system we have is no exception. There exists an “export” mechanism which gets you an “ASCII” dump of whatever stories you asked for, except it’s not really ASCII because it’s got all these custom high-bit characters in it that you have to deal with. Nope, its not UTF-8 or ISO-8859-1 either. Standards?!? What were you thinking?

But that’s not the hard part. In these days of HTML and XML and even standards based on XML like NewsML and NITF the notion that a print publication system would have and maintain it’s own proprietary markup language is mind boggling. But ours does. Modern notions like “close all open tags” have no meaning here.

Now the overall file format isn’t too bad, if you overlook the smattering of control characters scattered randomly throughout the data (never have figured that one out). It’s the content of the individual story bits that is a witch’s brew of bizarre tagging rules.

Submitted for your approval:

[TEXTOBJ]<USNEWS><NO1>
Web hed: Man bites dog
card/b1/mmedit
<NO>[BY]By Sam Spade
<MC>FOOTOWN GAZETTE WRITER
[TEXT]Lorum Ipsum blaa blaa blaa...


See, this is slick, you have markup tags within <...> brackets, and context-sensitive style “macros” between the [...] brackets. Tags can be “closed” by other tags or the classic </...> or the end of a block or nutso stuff like the <NO1>...<NO> above. That, by the way, is an “editor’s note” — be sure to strip all of those lest you let through damning internal comments. Good place for additional printable data too (um, not).

Some markup can reference config files, like <CFnn> means “change font” and will contain a number that references the font face specified in some INI file on the (UNIX) server somewhere (yes, an INI file).

When I say the style macros are context-sensitive I mean they essentially apply a set of <...> tags depending on their position. A [RAIL] in Sports may be completely different than a [RAIL] in Features… and lest we forget, a [...] tag is optional, you can always specify the <...> tags directly too. Ah, good times.

Oh, and don’t even get me started on the mixed DOS and Unix style line enders.

Thinking about running away screaming yet? I’m just getting started.

Field of Nightmares

Fielded data. Every web story database expects radical, forward-thinking things like headlines in headline fields and bylines in byline fields, etc. Well in our print system there are in-fact headline “objects” and there are even photo objects… and separate caption objects too (because separating photos and their captions makes sense… somehow).

There is however no separate delineation for bylines or datelines or anything else, because that’s all body copy. It’s just body copy with a different style applied, or maybe it’s a tag… or one of several, nested tags, or maybe a tag and a macro… whatever as long as it looks good in print.

On the upside, you get really good at Regular Expressions.

But wait, there’s more!

On the page layout, someone has to specifically mark each headline, story, photo and caption as belonging together. They need to be referenced together. This has no bearing on the printed page mind you so how often do you think this happens properly? Know references, know headlines; no references, no headlines.

Its my way or the highway

Ok, so you have your proprietary markup format, and your custom styles by section and publication. A tall order for even Perl’s legendary regular expression engine. Lets make it worse.

Dig it: you can have fundamentally different ways of doing things at different publications. Isn’t that cool?

Take headlines for instance: one of the sites I support uses a “headline object” for the main headline and another one for each subhead. This makes perfect sense until you find out that a another site uses a single headline object for all their headlines with the first one listed being the main headline.

No, problem, a little if...then action and… oh wait… a third site uses a headline object for the main headline and [...] styles within the body copy for subheads. Ok, now we’re into code hell.

When I suggested that some sites might want to change the way they do things… well, lets just say I’d have been better off suggesting boiled cat for lunch. “The way WE do it is better than the way THEY do it.”

Yeah, I’m sure it is. Would you like another helping of cat with that?

Lost in translation

Finally we have massaged and cajoled all this data onto whatever convoluted output format is being asked of us (usually some XML variant). We’ve converted and handled all the quirks and even mapped the print product’s lettered sections to the arbitrary section names used online (sortof).

It sucks and you know it. You strive for 80% of the content in the right place and you know you’re not there. No matter, you shove it off to the custom-built in-house web CMS (designed for newspapers!) and only then do you find out that the CMS has no notion of, oh I don’t know, how about stuff like “editions” or even freaking page numbers.

On top of that you get esoteric little nits like XML entity parsing errors. We spent a lot of time making things like m-dashes show up as — in the XML feeds only to find out that the parser being used by the CMS doesn’t handle that right at all and converts them to — which gets you a literal — on the page, which you might notice is not an m-dash.

That’s ok, there’s a bug report on it and it’s slated to be addressed in an update scheduled for Q1 2008, which is a nice way of saying, “fix it your own damn self.”

And in Q3 2008 when they roll in that fix, who’s pager you think is gonna go off?

So what’s my point

Most editors don’t care about any of this. They’re paying bottom dollar for a staff of 20-something-year-old upload monkeys to get their paper’s news online and rants over such obscure things as ASCII markup and XML entities are, to them, moot and irrelevant.

But my little rant isn’t just to vent my spleen (ok, maybe a little), it’s to illustrate a point. The entire technology foundation that many newspapers are built on today was never designed for web publishing (or any other non-newsprint-based publishing) and is in some cases so ill-suited to it as to become a huge road block to taking advantage of web-based opportunities. And I’m not just referring to editorial systems either, classified, advertising, billing… they all suffer like this.

Solutions become Rube Goldberg monstrosities duct-taped onto the outside of these legacy systems and we wonder why they dont work worth a damn. We wonder why our butts get handed to us on almost a daily basis. These systems are broken and no amount of duct-tape will fix them.

“Don’t give me a big mess of patches where each new patch you apply makes water squirt out somewhere else,” an editor once told me in a meeting about story exports to the web.

“But that’s what you already have!” I said.

I also went and imported an experimental Blogspot blog I had set up to blather on semi-anonymously about newspaper technical issues, called “Journo-Geek”. As it became really clear that I couldn’t fill one blog with interesting crap let alone two, I decided to nuke “Journo-geek” and incorporate it’s content here. WordPress has a nice import feature that made it pretty simple. If you care about such things, the Mass Media category houses most of those posts.

I’d also been hiding non-flattering posts about my day-job behind a login wall. Technically this was a pain because the WordPress plugin I was using to enable this was dodgy and quite frankly, the idea of hiding commentary clashed with what limited personal mores I may have (“Be Open, Be Vocal; Stay Open, Stay Vocal“).

So I looked through all the offending posts and edited them for clarity (and perhaps a little dulling of the sharper edges) and opened them back up. I did wind up deleting one or two (seriously, they were stupid) and I decided to keep two posts locked away (marked them “private” in WordPress) due primarily to personal content (but I still wanted a record of them).

What can I say, even the best jobs suck on some days.

“Um, where did Fresno go?”

Suddenly all the systems at The Fresno Bee dropped offline… followed by all the Fresno staffers I have in my IM buddy list disappearing. One of our guys was on the phone to someone down there and said, “I heard the fire alarm and then the line went dead.”

Holy crap!

It turns out that a fire sprinkler pretty much blew apart and flooded a small room where all the network and phones came into the building.

I wound up driving down that night because with the network out web updates — my domain — would be awfully tricky and while they would probably have the network back up sometime that night, if they didn’t by the time we knew for sure it would be too late to drive down and implement any wild hair ideas.

When I got there I immediately set up my laptop on the local network and popped in a Verizon EVDO card. Worst case I’d move updates to my laptop from the internal net and then bring up the EVDO card and move the updates manually to the web staging server.

Then I went to check out the damage. There had been 6″ to 8″ inches of water in this tiny little data room and most of the equipment had been either submerged or completely sprayed with this mucky 25-year-old sprinkler water. When I first walked in I didn’t have my camera, which was too bad because there was muck on the walls and guys were basically laying in it trying to get equipment pulled out to dry it off.

Then I sat in a pretty grim meeting of powerful people. The print operation was unaffected as all the internal systems and networks were fine, they’d still put out a paper. The major concern was the phones, the next morning people would be calling to place ads, to make complaints, to do any amount of business all via the phones. For an hour we discussed a variety of ways to get the phone traffic handled in the event they couldn’t get the phone system back up by morning.

Interestingly what was NOT discussed was the myriad of internet feeds than any newspaper has to put stuff on the web in various places including daily news feeds, classified feeds and the like. By exclusion, I got to see what really mattered.

So after the first edition went to bed, I dutifully juggled the data from one machine to another over the EVDO card and as expected 5 minutes later the network came up. While I called it a night, other folks were still at it using hair dryers to dry the remaining equipment.

Amazingly, they had most systems restored by morning, which is a testament to the hard work a lot of people did (but not me, I was in a hotel room working on the aforementioned system upgrade).

I need to keep my camera at the ready more.

Yesterday while leaving one of our remote offices I came across one of those guys that plasters his car with signs. Usually its just some extreme political viewpoint or other, but as I read this guy’s high art I just started laughing — then I frantically grabbed for my camera.

Let me get this straight, the judge ate your kids for breakfast… whaaaa?

Sadly I was on the slow side and there was traffic to deal with, so it’s not the best shot, but you get the point.

Then a few offramps up the freeway there was the kid in a toga on a Segway advertising pizza.

You heard me.

The problem is every newspaper is unique. Simply put, what works for one doesn’t necessarily work for another.

As an example, the papers I support were recently introduced to a new web content management system. The system isn’t really bad, although there are some latent issues, but every paper got — or will get — the exact same interface.

But every paper has subtle differences in the way they do things and this “standard” interface causes issues. It slows down production, forces workflow changes, and even in some cases forces product changes — all in the name of saving some nominal amount of money.

To avoid such nastiness, those papers lucky enough to have access to journo-geeks have them work up all manner of band-aids and workarounds. Yours Truly is certainly guilty of such hackery. The problem here is that who’s gonna support said band-aids if/when that journo-geek disappears? What if your journo-geek gets hit by the proverbial bus? Furthermore, what will happen when the offending system/application/whatever is updated? Will the band-aid update with it or will it blow apart requiring frantic emergency pages in the middle of the night?

The obvious (to me) solution is to shoot for “one size mostly fits all,” then provide ways for the journo-geeks and their ilk to tweak and modify things as they need. Make that way flexible yet scalable (upgradeable). And for crying out loud, make it documented.

Make it an interface so that programmers can modify your application.

Get it?

In a nutshell it sums up what I do. No, I don’t make neat videos, nor do I work at Yahoo or Flickr, but I understand at a very technical level how these web2.0y sites work, and more importantly their value (minus the hype), especially to certain companies.

Well, now wait. It’s not what I do right now. It’s what I haven’t been able to do despite my desire to.

It’s what I want to do.

Whoa.

It apparently sat on a shelf in his house for a number of years gathering dust (quite literally as I can attest to). Prior to that it was owned by a professional news photographer and it shows some substantial wear and tear. The serial number is quite early (6482469) so seems to indicates that this camera is most likely older than I am.

The one down side is that 3 batteries have leaked in the motordrive’s battery pack and are pretty much corroded in place. He had tried WD-40 to get them out (yikes).

So this thing is very dirty, very banged up, smells a bit funny… and is simply the coolest gift ever.