How many automated Barack Obama topic pages do we really need? That was the tweet posted by Scott Karp the other day. Our answer? None. With every news publisher jumping on the topic page bandwagon, it’s become a race to the bottom in terms of quality. Here’s how to make your pages stand out.

[Editors note: for a simple introduction to topic pages, read our topic page community guide first.]

Scott Karp wasn’t the first person to notice what I call the “topic page race to the bottom.” As every news organization scrambles to deploy their own version of automated topic pages, news readers are now inundated with too many topic pages that offer too little value. Not only are they starting to look the same, a lot of these pages are optimized for the GoogleBot (for SEO reasons) rather than human readers. In a quest to increase eyeballs and pageviews, we’ve forgotten the basic premise and goal of the topic pages: to provide readers with more context. So here are some tangible ideas to help your company create topic pages that stand out from the crowd, add value to the lives of your readers and still allow you to address your SEO and monetization goals.

1) Editorial Resources. First and foremost, we must abandon the idea that topic pages are fully automated and can scale easily without any staffing overhead. Yes, one of the attractions of these topic pages is the fact that they scale easily, but the idea that topic pages do not require human editors or curators is a fallacy. Pure and simple. If you don’t have the resources (either in house or outsourced) to manage and curate your topic pages, then this simply is not an effective solution for you. So the first step is to admit this and then if you are serious about your topic page strategy, allocate a sufficient number of resources to this project. (OK, we cheated and this first idea is not ‘a tangible step’ per se, but keep reading).

2) Curation. Once you have an editor or multiple editors, start curating these pages. The most successful topic pages combine efficient and intelligent aggregation with high quality human curation. What elements of a topic page can be curated by editors? Here’s a list:

1. A editorially curated introduction or synopsis: this is a no brainer. Instead of displaying an automated wikipedia bio, an editor should write a short snippet of text that address the fundamental question for all readers: why is this topic important to me? In this introduction, editors can include a short timeline of important events, a brief history of the topic, a easy-to-digest summary of why it’s important. Take a look at this NY Times topic page on health care reform. Not only is the summary kept up-to-date by an editor, in two or three sentences, I can understand exactly why this topic is interesting or important. Even though the summary goes on to provide much more detail, this first paragraph makes the difference between a good topic page and a great one. With all due respect to the Chicago Tribune, this topic page on Roland Burris gives me no introduction or context – I have no idea who he is, what he does or why he is important.
   
2. Individual articles:  if you think a reader can’t spot the difference between algorithmically generated articles and human curated ones, you are wrong. There is a place for both, but highlighting editorially selected articles can be a significant differentiator and really improve the overall quality of the page. These can be a handful of ‘Editor’s Picks’ that are kept up-to-date to guide readers to the must-read articles on any given topic. The key here is to ensure they are up-to-date. If you are worried, have the Editor’s Picks automatically dissapear after a certain amount of time has passed. Look at this topic page, where the Texas Tribune does a great job of highlighting important articles within the synopsis itself.
3. Multimedia: do not run keyword searches on Flickr or Youtube; the results are usually terrible. Partner with a provider who can guarantee relevant results for your pages, or editorially select individual videos and images. It’s better not to display any than show completely irrelevant (or even worse, inappropriate) images. While I was harsh on the Chicago Tribune above, I should point out that some of there topic pages do a great job with images and galleries, like the Barack Obama “life in pictures” gallery that sits at the top of this page:

3) Design Innovation: If you take a handful of automatic topic pages on the web, they all start looking the same. A list of articles, a bio, some images, and related topics. Even the layouts start looking the same. Work with your design team (or outsource it) to come up with some innovative designs or a compelling user experience. Here are some examples:

1. Using images: use high fidelity images to liven up the page. Or present articles or related topics as a newswall with images. Here’s an example of a newswall, even though it’s not within a topic page: http://search.independent.co.uk/
2. Change the layout: instead of presenting a long list of articles, think of changing the layout. Check out this ‘grid view’ from Daylife, who combine images and articles in an unorthodox view that provides a compelling visual experience.
3. Infographics work amazingly well. If you can’t design them in-house, link to ones from around the web. A lot of infographics allow embedding or you can at least display a low-res version and link to the original source. The A.P. has a great multimedia infographic about the recent Gulf Oil Spill, seen here on the PBS Site.

4) Structured data. Here we’re getting closer to the Semantic Web holy grail, but providing structured data can be hugely valuable. For a politician, provide his voting record. For a football player, his latest stats. For a company, stock information. For an actor, a list of his movies. You get the point. Integrating structured data need not be a manual process. With the help of the Freebase API (or 3rd party APIs) you can easily get structured data onto your site. While not technically structured data, if your organization has any unique data or content relevant to the topic, highlight it! For example, look at this Time.com topic page that shows relevant Time covers – this part of the page certainly caught my attention:

5) More context. Rather than displaying articles and blog posts, aggregate and link to other resources that provide extended context. This could include books, legislation, infographics, civic records, discussions and more. Perhaps more importantly, start linking to other websites, even if they are competitors. Be a part of the wider web, even if that means sending some traffic away (you will be repaid handsomely with more engagement, traffic and trust – a separate post will discuss that). Check out the middle column in this New York times topic page to see what I mean.

6) Q&A. While not exactly a topic page, Pro Publica presents context in the form of a Q&A about the Gulf Oil Spill. There’s a reason why FAQs are as old as the web itself: effectiveness.

7) Local. Let’s go back to Scott Karp’s original question of “how many automated Barack Obama topic pages do we really need?” While the answer might be “none,” what news readers really need are topic pages on the subjects that have local relevance. These are often (and most naturally) the most important topics and stories that affect their day-to-day lives. So instead of creating pages on Barack Obama and Iraq War, focus on creating extremely high-quality, laser-focused pages on issues of local relevance. Again, the Texas Tribune do a great job here. On their topic page directory, almost all the highlighted topics are local, and extremely relevant to their core audience.

8. Avoid death by keywords. It’s extremely tempting to automate topic pages by looking for keywords in your archived articles. But there are many false positives (“My favorite fruit is the Apple” shows up on your page about the company Apple) and even more false negatives (your page on “Climate Change” misses all your articles that reference “global warming”).  There are many ways to solve this problem. If your editors are disciplined about tagging, those can help. Using a third party tool like OpenCalais to extract topics from articles can help. Topic page providers usually have software to disambiguate and extract topics with a high degree of precision. Semantic technology and natural language processing can help solve the problem almost completely, but few can afford this. The best answer is usually a smart combination of all the above. Also, choosing topics wisely can help cut down on false positives (i.e. a topic on Associated Press in a system using keyword search will likely result in thousands of articles each day due to the bylines).

Steve Yelvington, who is a digital strategist at Morris Communications has a fantastic blog where he has a great post on topic pages and how to tailor these pages for the right audience. He covers a lot of what I mention here, and I urge you to hop over and read his post. But I’m going to steal the last two paragraphs of his post and embed them here, because I think they capture the crux of my thesis. The last point (yes, about awesome hip-hop) is especially important:

Done well, the topics page provides the casual, occasional user with a gentle, almost encyclopedic introduction to the topic (public issue, person, place, thing). But the regular, loyal user benefits too.

Done poorly — and I’ve looked recently at some topics pages that would curl my hair, if I had enough left to curl — a topics page leaves both loyal and occasional users with one of those “WTF” moments.

The biggest dangers come from these sources:

1) Lack of a synopis that makes sense. Some sites don’t even both writing a synopsis. Others seem to have assigned the work to interns from the marketing department.

2) Misplaced trust in automation. I found a USA Today topics page about the BBC. A bot had assembled it. Every oblique mention of the BBC was churned up. The page made no sense at all. If I want to run a search, I’ll go to Google, thank you.

3) Inflexible formatting. A format or template should be a starting point, not an ending point. If your community has an awesome hip-hop culture, your hip-hop page should be awesome and hip-hop.”

With that excerpt from Steve, we’ll wrap up this post and open up the comments to suggestions from the community. I’ll follow up with a second post with your crowd-sourced ideas and suggestions to help escape the topic page race to the bottom.