While I’m somewhere between Brazil and Argentina, on my vacations, I’ll let you with this interesting article about the “readability” of the 3 major books of Tolkien: The Lord of the Rings, The Hobbit and The Silmarillion.
Which one of them is the hardest to read? Which one can a person who’s not a fan find it a bit boring?
Let’s not guess answers here and analyze data! Read the article below:
Visualizing Tolkien
Why?
First and foremost, I am a huge fan of J.R.R. Tolkien’s work. I have lost count of the number of times I have read
Lord of the Rings. But I had not readThe HobbitorThe Silmarillionyet, and decided to put an end to that situation.Prior to purchasing it, I read several reviews about
The Silmarillion. One of the reviewers argued that it was the hardest book to readbecause ‘and’ was the most used word in the book. I wondered if that was the case. And if not, why is it thatThe Silmarillionis so hard to read? And I can testify that it is definitely hard to read: I attempted to read it at least thrice past year, but I ended reading several other books instead,Lord of The Ringsagain as well.The classic graphs
To find out if ‘and’ is the most frequent word in
The Silmarillion, I wrote a simple program who counted how many times did each word appear in the book. This quickly contradicted the affirmation by that reviewer, since the most frequent word inThe Silmarillionis the, followed by and and of.Obviously, as I had this program and it could analyse any text instantly, I thought that maybe I could analyse the other two main important works from Tolkien:
The HobbitandThe Lord Of The Rings.If I place all results side by side I may be able to deduct why ‘The Silmarillion’ is not as readable as the others, I said to myself. So I did:
Interestingly enough, all three books share the same top three words. In fact, all of their top words are pretty much the same (the, and, of, in, to, he, that, …). So it was totally unfair (apart from incorrect) to blame them for the lack of readability of a book.
What about the proportions and the distribution of words? If we compare the shapes of each chart together, it is easy to see that while the shapes for
The HobbitandThe Lord of The Ringscharts are really similar, the same does not occur withThe Silmarillion, where there is a huge quantitative difference between the top three words and the rest. Now that might explain something!But I was not satisfied with this analysis yet. You cannot reduce style differences to numbers only; there were a number of factors that I had not considered yet: relations between words, typical constructions, language richness, even the length of the text itself! So I built a few more charts:
The word count chart confirms something we knew:
The Hobbitis shorter thanThe Lord of The Rings, but slightly surprises me when it shows so clearly thatThe Silmarillionis almost half the length thanLOTR. Specially because the reader does not experience that very same perception.Maybe the readability differences could be attributed to the originality index? That is an index that I “invented”, taking the number of unique words for each book and dividing it by the total word count. That would provide us with another way of comparing the books. But the originality index chart is surprising as well. I expected
The Hobbitto have the lowest index, since that was the book that I perceived as easiest to read; in fact I even thought it was slightly dumb at certain points, too much children-oriented. But I was wrong; proportionally it is the most original book, and according to this chart,The Silmarillionwould be only a bit less enjoyable thanLOTR, that with only a 3% index, should be a bore.My assumptions were not working, because
LOTRis not a bore!Could it be that I had taken into account the stop words but I should have not? I am referring to common English words such as the, and, of… – which are the most frequent in these works! On one hand I was very tempted to execute again the program, excluding those words. On the other hand, I did not believe it could be a good idea, since when we read a book, we are reading the stop words as well. We are not one of those rudimentary search engines who need to filter information out in order to distinguish keywords! If I removed those words from the text, the results would correspond to entirely different books.
Still, I decided to build a simple chart comparing the proportion of stop words vs. non stop words. Again, the results were surprising. One would expect
The Silmarillionto have more filler text, but it was quite the contrary, withThe Hobbitbeing the richest in stop words. In any case, the differences between books were not very significative.I ran another quick test (not pictured in this page) where I built these charts for
Draculainstead ofLOTR. That returned a very different set of results on every chart, so maybe instead of using these indices to compare books of the same author, they could be used to compare books of a known authors versus anonoymous books — that way we could guess who was the author of a book or piece of text!And here end the most classical-academic of my speculations about Tolkien. I was satisfied with refuting that reviewer regarding the overuse of ‘and’, and had also found some interesting surprises. I could think of more indices to be calculated: the proportion of verbs, adverbs, adjectives, nouns and etc; types of used tenses, type of constructions… but if I really wanted to get serious with this whole text analysis business, that would require way more time and resources than building a few charts and speculating about them.
Well, well, well…next time someone criticizes the Silmarillion (my favorite book EVER) saying it’s hard to read, boring story and some other absurds like that, here it’s provided the empirical data proving that’s not the case! Full objectiveness and zero subjectivity!
By
