CiteIt.net

Building trust in media

Creating the text version of an HTML page

by

When citing sources, CiteIt removes all HTML formatting before computing the 500 characters of context before and after each selection.

But although it is expedient to remove HTML tags, so as to simplify the text extraction process, it would be desirable for the text version to maintain white space and some other formatting.

Example:

Suppose you had an example series of ordered and unordered lists which mimic the “Step 3: Profit” meme:

How to get rich with a dot com company:

ul

li Sell dog food on the internet.

li ????

li Profit

A simple stripping of the formatting would remove the list styling.

It would be preferable to

  • Indent the list, and
  • Put each item on its own line.
  • Ordered lists should start with a number and unordered lists with an asterisk.

Existing Programs:

There is an existing Java library called Boilerplate that runs on Google App Engine. It doesn’t quite do what we want, but is similar.

Do you know of a library or approach which could make it possible to create the type of text version desired?

A similar process needs to be done for other document types such as PDF and Doc.

First Feature Requests and Bug Reports

by

I got my first user feedback on the wordpress plugin from my friend Daniel Miller.  In the interests of documenting feedback here is Daniel’s feedback:

Feature: Add neotext to the Preview/Publish system so that the user doesn’t have to visit api.CiteIt.net and submit a new url.  I think this is possible, I just have to research the WordPress API.  One optimization I’d also like to do is detect whether a “cite” attribute has been used so the webservice doesn’t run unnecessarily.

Bug: Fix the extra non html code that gets included in the text:

The expandable before-context of the quotation I pulled from neotext.net  starts with “.. /analytics.js’,’ga’); ga(‘create’, ‘UA-65403609-1’, ‘auto’); ga(‘send’, ‘pageview’);

Feature: enable the editing (and possible formatting) of the quote

Question: Say I neotextify a page and then later I decide I don’t want the expanding quote contexts in that article. Is there an easy non-technical way I can turn off the neotext plugin for just one page in WordPress?

Answer: You can remove the neotext by removing the “cite” attribute from the html.  Perhaps there could be a way to do this with the GUI.

Bug: The headings in the context for my quote are run together with no space between the surrounding text and the heading.

Possible solution: This appears to be the result of line wraps that don’t contain a space at the end of them.  This will require a different way of generating the text-version of the html

Feature: The context expansion arrows should flip to point the opposite direction when expanded.

Feature: Is there supposed to be a link to the original document in the top expandable section? Or maybe at the bottom of the bottom expandable section? Seems like that might be a nice feature, although maybe some people wouldn’t want that, so maybe simpler is better? (thinking out loud here)

Feature: Some of the javascript files included by the plugin are not minified. Would be nice if they were.

More Feedback:

If you have further suggestions send them my way.  My gmail account is timlangeman.  If I like them, I’ll have you add them to the Github issues pages.