CiteIt.net

Building trust in media

How to Use Google Speech to Text to Generate a Transcript from a Youtube Video

by


Here’s how to generate transcripts with the Google Speech to Text api.

I’m providing instructions for running this on a micro Debian instance on Google Cloud Compute Engine.

Here’s how to download an interview Tyler Cowen did with Malcolm Gladwell.

1) Install YDL (The Youtube Download Library) for Commandline

The apt-get option seems to have an outdated version of ydl:  (I’m using Debian)

Alternate YDL Option:

If you can’t run youtube-dl because debian has on old version, try:

sudo apt install python3-pip
sudo pip3 install --upgrade youtube-dl

2) Get the FFMeg library

sudo apt-get update
sudo apt-get install ffmpeg

3) Running the YDL Download Command

youtube-dl --extract-audio --audio-format wav --audio-quality 5 --postprocessor-args "-ac 1" https://www.youtube.com/watch?v=ehlhrqSWPbo > malcolm_gladwell.wav

4) Copy .wav file to Storage Bucket

install gcsfuse to mount bucket

gsutil cp malcolm_gladwell.wav gs://bucket_name/

 

5) Upload filename.wav to Speech-text API

gcloud ml speech recognize-long-running \
'gs://citeit_speech_text/malcolm-gladwell.wav' \
--include-word-time-offsets \
--language-code='en-US' \
--async
Check Job Progress
config = types.RecognitionConfig(
sample_rate_hertz=44100,
enable_word_time_offsets=True,
audio_channel_count=2,
language_code='en-US'

 

6) Download transcript.json

sudo pip3 install --upgrade webvtt-py

import webvtt

for caption in webvtt.read(downloads/5BXtgq0Nhsc.en.vtt'):
    print(caption.text)

 

7) Create timings.json

 

8) Create a Demo Page

 

Cherry-Picked Quotations on the Jefferson Memorial

by


Jefferson Memorial: Panel Three (Northeast Portico)

6 Separate Quotations: Northeast Portico

Although it looks as though the text etched into Panel Three of the Jefferson Memorial is a single quotation from Jefferson, it actually is a compilation of 6 separate quotations.  Can you see what was lost when the context of each excerpt was stripped away?

Notice the text two-thirds of the way down: “

Nothing is more certainly written in the book of fate than that these people are to be free.

 

Jefferson Memorial: Panel 3

Source: Wikipedia Commons.

Individual Quotes:

These Individual Quotes are broken out on the official Monticello website into 6 separate quotations, although quotation #4 about the slaves’ freedom stops short of mentioning their deportation:

      1. “.. But let them [members of the parliament of Great Britain] not think to exclude us from going to other markets, to dispose of those commodities which they cannot use, nor to supply those wants which they cannot supply. Still less let it be proposed that our properties within our own territories shall be taxed or regulated by any power on earth but our own. The god who gave us life gave us liberty at the same time: the hand of force may destroy, but cannot disjoin them.” – “A Summary View of the Rights of British America“
      2. “.. For in a warm climate, no man will labour for himself who can make another labour for him. This is so true, that of the proprietors of slaves a very small proportion indeed are ever seen to labor. And can the liberties of a nation be thought secure when we have removed their only firm basis, a conviction in the minds of the people that these liberties are the gift of God? That they are not to be violated but with his wrath? Indeed I tremble for my country when I reflect that God is just: that his justice cannot sleep for ever . . . .” – Notes on the State of Virginia, Query XVIII1
      3. .. “The whole commerce between master and slave is a perpetual exercise of the most boisterous passions, the most unremitting despotism on the one part, and degrading submissions on the other. Our children see this, and learn to imitate it. . . .” – Notes on the State of Virginia, Query XVIII2
      4. .. The principles of the amendment however were agreed on, that is to say, the freedom of all born after a certain day, and deportation at a proper age. But it was found that the public mind would not yet bear the proposition, nor will it bear it even at this day. Yet the day is not distant when it must bear and adopt it, or worse will follow. “Nothing is more certainly written in the book of fate than that these people are to be free. Nor is it less certain that the two races, equally free, cannot live in the same government. Nature, habit, opinion has drawn indelible lines of distinction between them. It is still in our power to direct the process of emancipation and deportation peaceably and in such slow degree as that the evil will wear off insensibly, and their place be pari passu filled up by free white laborers. If on the contrary it is left to force itself on, human nature must shudder at the prospect held up. We should in vain look for an example in the Spanish deportation or deletion of the Moors.” – Jefferson’s Autobiography
      5. .. “Preach, my dear sir, a crusade against ignorance; establish & improve the law for educating the common people.” – Jefferson to George Wythe, August 13, 1786
      6. .. “It is an axiom in my mind that our liberty can never be safe but in the hands of the people themselves, and that too of the people with a certain degree of instruction. This it is the business of the state to effect, and on a general plan.” – Jefferson to George Washington, January 4, 1786

In Defense of Jefferson:

Some people argue that Historians should throw Jefferson out of the American pantheon for his views on race.  Benjamin Schwarz argues in a March 1997 article in The Atlantic that this would be a mistake:

  1. Even the worst parts of the past are connected to others, and
  2. It “would deprive the United States of the figure central to what is singular and most admirable about the promise of American life — a promise that is already largely forgotten.

Southeast Portico: Open to “progress of the human mind” and “enlightenment”

Jefferson was a complex man.  What he takes away with one hand, he gives with another.

On the opposite wall from the Northeast Portico, the Southeast Portico provides another quote from Jefferson — an edited version of a letter to Samuel Kercheval (June 12, 1816) — that indicates that Jefferson was open to changing his mind and the law.  Here is the original, without the editing:

Source: Letter to Samuel Kercheval

Thomas Jefferson | July 12, 1816

 

I am certainly not an advocate for frequent and untried changes in laws and constitutions. I think moderate imperfections had better be borne with; because, when once known, we accommodate ourselves to them, and find practical means of correcting their ill effects. But I know also, that laws and institutions must go hand in hand with the progress of the human mind. As that becomes more developed, more enlightened, as new discoveries are made, new truths disclosed, and manners and opinions change with the change of circumstances, institutions must advance also, and keep pace with the times. We might as well require a man to wear still the coat which fitted him when a boy, as civilized society to remain ever under the regimen of their barbarous ancestors.

View Photo: Southeast Portico

In the Jefferson Memorial, the southeast panel advocating laws and institutions change with “progress of the human mind” appears opposite the northeast panel with the cherry-picked quote about deportation. View the full-sized panoramic photo of the Memorial to inspect the inscriptions and see both panels side by side.

Jefferson Memorial

Jefferson Memorial, Washington, DC.  Source: Wikiquote

 

Background Reading: Seeing White Podcast Series: Episode 4

  1. Notes, ed. Peden, 163.  Manuscript available at Massachusetts Historical Society.↩

  2. Notes, ed. Peden, 163.  Manuscript available at Massachusetts Historical Society.↩

New Version Fixes Some Bugs

by


I released an updated version of the wordpress plugin tonight.  This version handles the filename hashing a little bit differently.  It escapes a list of characters such as apostrophes and quotation marks that can be problematic.

The basic functionality is now working pretty well, but the performance needs to be improved for pages that include multiple citations on a single page.  The web service also has a memory leak that needs to be patched before it is really ready for prime time.

Dear Medium, You Could Combat Fake News by Showing More Quote Context

by


Inspiration:

Some Americans have become so disillusioned with the mainstream press that they refuse to trust basic reporting,  assuming that even direct quotations may have been taken out of context.

Cherry-picked quotations are an old problem, but one which digital tools are uniquely suited to combat.  In this article I’m going to demonstrate how a new set of authoring tools I’ve developed can allow authors to establish greater credibility, by providing readers the additional context they need to quickly evaluate a quotation’s context.

Take for example, the following cherry-picked quotation about slavery from Thomas Jefferson’s autobiography:

Nothing is more certainly written in the book of fate than that these people are to be free.

"Jefferson

The Jefferson Memorial

In this selective quotation, Jefferson appears to be a foreword-looking man, anticipating the emancipation of African-Americans. But if you click on the above down arrow, you can see that the text expands to reveal subsequent sentences which describe Jefferson’s desire to deport all the freed slaves, a part of history which is often omitted from our founding  myths. 1

Quotations that appear in print are frequently devoid of context, requiring the reader to trust the author or follow a footnote.  I call these “severed quotations” because the context necessary for readers to evaluate them has been chopped off.  I use the term “severed” as a way to suggest that we need to establish a new norm for quotations — a norm which the digital medium has only now made possible, where the text surrounding a quotation can be expanded to provide readers with a fuller context.  This new norm can help readers distinguish between those authors who desire to live to a higher standard of accountability, from those who play fast and lose with the record.

Out of Context Quotations in the Media

Take for example, the  Saturday Night Live Sketch that Parodied Sarah Palin:

I can see Russia from my house!

This line was actually a exaggerated paraphrase from Sarah Palin that makes more sense if you read the original ABC News interview’s full transcript.

ABC New’s Actual Interview:

The interview was conducted in Alaska; and early in the interview ABC anchor Charles Gibson raised the topic of Russia’s proximity to Alaska:

GIBSON: Let’s start, because we are near Russia, let’s start with Russia and Georgia.

Later in the exchange, Alaska’s proximity comes up again, with Palin answering Gibson, by saying:

PALIN: They’re our next door neighbors and you can actually see Russia from land here in Alaska, from an island in Alaska.

I was not a Palin supporter, but I am sympathetic to conservatives who say that, whatever faults Palin may have had, ABC News and Saturday Night Live treated her unfairly by taking an otherwise innocuous statement out of context.  It is an accumulation of many of these sorts of incidents that degrades conservatives’ trust in mainstream media sources like ABC News.2

If media outlets want to distinguish themselves from the competition, one visible way of doing this would be to adopt the sort of expanding quotations I’ve demonstrated and coupling this with a campaign to regain public trust.

I created the open-source Neotext project as a way to demonstrate the technical feasibility of this form of expanding quotations, and as a project to develop working implementations on many platforms.

You can experience what it is like to create your own Neotext quotations on my demo site by clicking on the “Test Drive” button below.

Test Drive: WordPress Plugin  (alpha)

 

Popularizing Neotext

The goal of the Neotext project is to provide authors with tools that allow their readers to make an informed evaluation of sources, and to inspire greater confidence in the works of authors who are willing to hold themselves to a higher standard of accountability.

How Medium could adopt Neotext

Medium could be the first platform to adopt this digital writing feature, if it:

  1. added a source URL field to their blockquotes and
  2. incorporated a citation web service, like the one I developed.

How to Help This Project:

If you don’t work at Medium.com and want to help on the WordPress plugin, I’m looking for:

  • Programming collaborators to help me improve the code, and
  • WordPress authors who are interested in being alpha-testers of my plugin.

If you’re interested in staying up to date with the latest news, join my email newsletter or subscribe to the RSS feed.

How the Code Works:

The plugin I created takes advantage of standard html’s blockquote cite attribute, where the url is specified with the “cite” attribute.  Platforms like Medium would need to modify their editors to allow the author to specify a URL with their blockquotes:

  1. When the article is published, the Web Authoring Tool (either WordPress or Medium) scans published articles and checks to see if the article contains a <blockquote cite= “url”> tag.  If the article does, it uses the HTTP POST command to programmatically send the article’s url to the Python web service I created.
  2. The Python Web Service gets the submitted url, downloads the specified article, and for every cited blockquote url it finds:
    • it retrieves the cited document,
    • creates a text-version of the cited source, and
    • locates the quotation starting position.
  3. After the web service locates the quotation, it copies the 500 characters of text before and after the quotation.
  4. The web service saves the 500-character contextual text snippets to a json file on Amazon S3, constructing an identifier from a hash of the quoted text and the page’s url.
  5. When a reader views the citation on the published page, their browser:
    • uses jquery to find any <blockquote> tags with the “cite” attribute,
    • uses javascript to compute the quote’s hash identifier (from citing_url, citing_quote, and cited_url),
    • looks up the json file in the Amazon cloud, and
    • injects the json content into hidden <divs> on the citing page and displays arrows above and below each <blockquote>

Related: Download Sample HTML template

Related articles:

  1. Journalists and Technologists Should Collaborate to Build More Trustworthy Media, which I also wrote.
  2. Review: Ted Nelson’s Philosophy of Hypertext: I was inspired while writing this article to create Neotext, as a way to evoke and eventually transition to a hypertext more like Nelson’s original vision.

Contact me if you’re interested in helping with the programming or UI, or check out the project website or Github code.

 

  1. This Jefferson quotation can be found in multiple sources, including the Jefferson Memorial, but if we draw in the full context of the quote from page 68 of Jefferson’s 1821 autobiography, we find that additional context reveals a more complicated history.↩

  2. Distrust is also caused by conservative outlets continual promotions portraying the competition as unfair and biased.↩

Article: How Neotext could prevent some types of Fake News

by


Last month I posted an article I wrote about how Fake News, in the form of Out-of-Context Quotations, is not a new issue, but the internet poses both challanges and opportunities.

I described 2 examples of how media sources in 1999 and 2008 took quotations by Al Gore and Sarah Palin out of context and how this could be prevented if media outlets used Neotext.

I also expanded the concept of quote context to video and laid out 3 challenges to technologists.

Read Article

Progress Report: Hash Algorithm Issues

by


Here’s a progress report on where things stand.

I’ve been working on the webservice code to try to fix some problems with how the quote hash values are created.  Right now the hash is computed in two places – by the client, using javascript, and by the python web service.

Most of the time both sets of code produce the same result, but there are some cases when they do not, which prevents the client from finding the generated json file.

Before I go too much further, I’d also like to switch the hash algorithm from sha1 to something like sha256.  This should be a fairly simple switch because I had planned for the hash algorithm to be swappable, without breaking backwards compatibility.

 

First Feature Requests and Bug Reports

by


I got my first user feedback on the wordpress plugin from my friend Daniel Miller.  In the interests of documenting feedback here is Daniel’s feedback:

Feature: Add neotext to the Preview/Publish system so that the user doesn’t have to visit api.CiteIt.net and submit a new url.  I think this is possible, I just have to research the WordPress API.  One optimization I’d also like to do is detect whether a “cite” attribute has been used so the webservice doesn’t run unnecessarily.

Bug: Fix the extra non html code that gets included in the text:

The expandable before-context of the quotation I pulled from neotext.net  starts with “.. /analytics.js’,’ga’); ga(‘create’, ‘UA-65403609-1’, ‘auto’); ga(‘send’, ‘pageview’);

Feature: enable the editing (and possible formatting) of the quote

Question: Say I neotextify a page and then later I decide I don’t want the expanding quote contexts in that article. Is there an easy non-technical way I can turn off the neotext plugin for just one page in WordPress?

Answer: You can remove the neotext by removing the “cite” attribute from the html.  Perhaps there could be a way to do this with the GUI.

Bug: The headings in the context for my quote are run together with no space between the surrounding text and the heading.

Possible solution: This appears to be the result of line wraps that don’t contain a space at the end of them.  This will require a different way of generating the text-version of the html

Feature: The context expansion arrows should flip to point the opposite direction when expanded.

Feature: Is there supposed to be a link to the original document in the top expandable section? Or maybe at the bottom of the bottom expandable section? Seems like that might be a nice feature, although maybe some people wouldn’t want that, so maybe simpler is better? (thinking out loud here)

Feature: Some of the javascript files included by the plugin are not minified. Would be nice if they were.

More Feedback:

If you have further suggestions send them my way.  My gmail account is timlangeman.  If I like them, I’ll have you add them to the Github issues pages.