In order to test the package I retrieved the titles of the XKCD web comics included in my RXKCD package and produced a word cloud based on the titles' word frequencies calculated using the powerful tm package for text mining (I know, it is like killing a fly with a bazooka!).
library(RXKCD)
library(tm)
library(wordcloud)
library(RColorBrewer)
path <- system.file("xkcd", package = "RXKCD")
datafiles <- list.files(path)
xkcd.df <- read.csv(file.path(path, datafiles))
xkcd.corpus <- Corpus(DataframeSource(data.frame(xkcd.df[, 3])))
xkcd.corpus <- tm_map(xkcd.corpus, removePunctuation)
xkcd.corpus <- tm_map(xkcd.corpus, tolower)
xkcd.corpus <- tm_map(xkcd.corpus, function(x) removeWords(x, stopwords("english")))
tdm <- TermDocumentMatrix(xkcd.corpus)
m <- as.matrix(tdm)
v <- sort(rowSums(m),decreasing=TRUE)
d <- data.frame(word = names(v),freq=v)
pal <- brewer.pal(9, "BuGn")
pal <- pal[-(1:2)]
png("wordcloud.png", width=1280,height=800)
wordcloud(d$word,d$freq, scale=c(8,.3),min.freq=2,max.words=100, random.order=T, rot.per=.15, colors=pal, vfont=c("sans serif","plain"))
dev.off()
As a second example, inspired by this post from the eKonometrics blog, I created a word cloud from the description of 3177 available R packages listed at http://cran.r-project.org/web/packages.
require(XML)
require(tm)
require(wordcloud)
require(RColorBrewer)
u = "http://cran.r-project.org/web/packages/available_packages_by_date.html"
t = readHTMLTable(u)[[1]]
ap.corpus <- Corpus(DataframeSource(data.frame(as.character(t[,3]))))
ap.corpus <- tm_map(ap.corpus, removePunctuation)
ap.corpus <- tm_map(ap.corpus, tolower)
ap.corpus <- tm_map(ap.corpus, function(x) removeWords(x, stopwords("english")))
ap.tdm <- TermDocumentMatrix(ap.corpus)
ap.m <- as.matrix(ap.tdm)
ap.v <- sort(rowSums(ap.m),decreasing=TRUE)
ap.d <- data.frame(word = names(ap.v),freq=ap.v)
table(ap.d$freq)
pal2 <- brewer.pal(8,"Dark2")
png("wordcloud_packages.png", width=1280,height=800)
wordcloud(ap.d$word,ap.d$freq, scale=c(8,.2),min.freq=3,
max.words=Inf, random.order=FALSE, rot.per=.15, colors=pal2)
dev.off()
As a third example, thanks to Jim's comment, I take advantage of Duncan Temple Lang's RNYTimes package to access user-generate content on the NY Times and produce a wordcloud of 'today' comments on articles.
Caveat: in order to use the RNYTimes package you need a API key from The New York Times which you can get by registering to the The New York Times Developer Network (free of charge) from here.
require(XML)
require(tm)
require(wordcloud)
require(RColorBrewer)
install.packages(packageName, repos = "http://www.omegahat.org/R", type = "source")
require(RNYTimes)
my.key <- "your API key here"
what= paste("by-date", format(Sys.time(), "%Y-%m-%d"),sep="/")
# what="recent"
recent.news <- community(what=what, key=my.key)
pagetree <- htmlTreeParse(recent.news, error=function(...){}, useInternalNodes = TRUE)
x <- xpathSApply(pagetree, "//*/body", xmlValue)
# do some clean up with regular expressions
x <- unlist(strsplit(x, "\n"))
x <- gsub("\t","",x)
x <- sub("^[[:space:]]*(.*?)[[:space:]]*$", "\\1", x, perl=TRUE)
x <- x[!(x %in% c("", "|"))]
ap.corpus <- Corpus(DataframeSource(data.frame(as.character(x))))
ap.corpus <- tm_map(ap.corpus, removePunctuation)
ap.corpus <- tm_map(ap.corpus, tolower)
ap.corpus <- tm_map(ap.corpus, function(x) removeWords(x, stopwords("english")))
ap.tdm <- TermDocumentMatrix(ap.corpus)
ap.m <- as.matrix(ap.tdm)
ap.v <- sort(rowSums(ap.m),decreasing=TRUE)
ap.d <- data.frame(word = names(ap.v),freq=ap.v)
table(ap.d$freq)
pal2 <- brewer.pal(8,"Dark2")
png("wordcloud_NewYorkTimes_Community.png", width=1280,height=800)
wordcloud(ap.d$word,ap.d$freq, scale=c(8,.2),min.freq=2,
max.words=Inf, random.order=FALSE, rot.per=.15, colors=pal2)
dev.off()



The post is interesting and I could replicate your second example. But I don't know how to do it if I have a text in a txt file or a word file. Your example just works with a html table but very often we have whole texts. I will be very grateful if you can make a world cloud using a txt file.
RispondiEliminaNoam
Dear Noam,
RispondiEliminaYou can find both the answer to your question and a nice introduction to text mining in R in the vignette of the tm package:
install.packages("tm")
library("tm")
vignette("tm")
HIH!
Thank you Paolo, I'm going to read about the tm package. This is my first meet with text mining bacause I just use R for my classes of statistics.
RispondiEliminaNoam
You are welcome Noam!
RispondiEliminaI'm not a text mining expert as well but the tm package seems to provide a collection of tools that can be useful for solving both basic and more advanced problems in this interesting field.
wow *_*
RispondiEliminaVery nice. Liked it and probably will use it.
RispondiEliminaA great example. However sometime in the past two weeks back from 2011/11/30 the directory and file was removed. So "http://cran.r-project.org/web/packages/available_packages_by_date.html" is not found because the "packages" directory is no longer there. How about using another web site as an example?
RispondiEliminaThanks for the update! Feel free to suggest a web site of interest as an alternative.
RispondiEliminaThanks Paolo, this might be impossible but how about any text on a basic news web page like www.washingtonpost.com. Ignore pictures and pick phrases/sentences based on commas, periods and breaks "-". We could manually input a target web page and the example would wordwrap the page.
RispondiEliminaThanks, Jim
Thanks Jim for your suggestion! I have updated the post in accordance with your advice (more or less).
RispondiEliminahow does one increase the plotted area of the word cloud...by increasing the the dimensions of the png image, I am only getting a bigger image..with most of it being blank white space, and a small word cloud at the middle of the plot
RispondiEliminaIt seems that this problem (bug?) is related to the graphics device driver you decides to use (pdf, svg, png, etc.). I think that Ian Fellows, the author of the package, could answer your questions more appropriately than me!
RispondiEliminaHi,
RispondiEliminayou can add some really nice colors to your word cloud using the Free color brewer color rules. download their spreadsheet to add the capability into your code. see colorbrewer2.org
Thanks for stepping in. The RColorBrewer package allows users to access the beautiful and well conceived colorbrewer palettes from R, take a look at it!
RispondiEliminaThis sounds very promising. I didn't know that R let you create this kind of visualizations. I used wordle in the past, and R for a bioinformatic project at the University.
RispondiEliminaThanks for the post..when i run the second example nothing happens...just checking if the png wordcloud files should be in any particular folder
RispondiEliminaThanks
Arun David
Dear Arun,
RispondiEliminaI checked the code for the second example and it seems to work without a hitch. If you have used the exact same code presented in the post, the image should be generated in your working directory and named wordcloud_packages.png.
HIH!
I am a beginner with R. How can I create a "phrase" cloud? Basically I have a list of 100 strings/phrases which i want to present as a cloud.
RispondiEliminaI have difficulty understanding the goal of your exercise. A word cloud is a visual tool which can help in perceiving the most prominent (frequent) terms in a collection of words using either color or size. In your case I can imagine your phrases are all different among them; therefore a representation based on frequency make little sense to me.
RispondiEliminaThe list of phrases has been derived based on importance; hence it is in order of importance. Is it possible to represent them like a cloud? How?
EliminaMaybe give the phrases descending frequencies?
More importantly, the examples above are for single words. What about phrases?
Try something like this:
RispondiEliminaPut your data in a file phrases.txt with a single phrase for each raw:
all work and no play
makes jack
a dull boy
1) Import the phrases in R
my.phrases <- scan("phrases.txt", what="char", sep="\n")
2) import or create a vector with the frequencies (convert your order of importance in same way to frequencies), e.g.
my.freq <- c(10,20,15)
3) Plot the wordcloud
library(wordcloud)
wordcloud(my.phrases, my.freq)
thanks a million!
EliminaYou are welcome!
EliminaIs there a limit for the word cloud? Just a handful of items are visible. I set frequency for every 10 items as the same; hence, 10 for first 10 items, 9 for next 10 items...
EliminaSee ?wordcloud. Take a look at min.freq and max.words arguments.
RispondiEliminahi this is good !! but it depends on word frequency,i will try on sentiment wordcloud along with frequency it means positive word show different color and negative one show different color.can any one help me.
RispondiEliminaHi Paolo, I was able to build word cloud from csv file with little modification of above code. Would like to know if it is possible to create different shape like what tagxedo provides?
RispondiEliminaDear Hari, from what I can see from the help page this feature is not currently available in the wordcloud package. You could suggest it to the author.
RispondiEliminaFYI
RispondiEliminaHello Mr.Ian,
I am using the package Word Cloud authored by you in R. The package creates a word cloud in a circle shape, would like to know if it is possible to make different shapes of word cloud like what tagxedo provides.
Dear Hari, in order to contact the author of the wordcloud package you should use the information you can find at http://cran.r-project.org/web/packages/wordcloud/index.html
RispondiEliminaI am not related to the author of this package nor have any connection with him.
Great post!! Super useful!
RispondiEliminaI get a lot of errors on this:
RispondiEliminaError: failed to load external entity "http://cran.r-project.org/web/packages/available_packages_by_date.html"
> ap.corpus <- Corpus(DataframeSource(data.frame(as.character(t[,3]))))
Error in t[, 3] : object of type 'closure' is not subsettable
> ap.corpus <- tm_map(ap.corpus, removePunctuation)
Error in tm_map(ap.corpus, removePunctuation) :
object 'ap.corpus' not found
> ap.corpus <- tm_map(ap.corpus, tolower)
Error in tm_map(ap.corpus, tolower) : object 'ap.corpus' not found
> ap.corpus <- tm_map(ap.corpus, function(x) removeWords(x, stopwords("english")))
Error in tm_map(ap.corpus, function(x) removeWords(x, stopwords("english"))) :
object 'ap.corpus' not found
> ap.tdm <- TermDocumentMatrix(ap.corpus)
Error in TermDocumentMatrix(ap.corpus) : object 'ap.corpus' not found
> ap.m <- as.matrix(ap.tdm)
Error in as.matrix(ap.tdm) : object 'ap.tdm' not found
> ap.v <- sort(rowSums(ap.m),decreasing=TRUE)
Error in is.data.frame(x) : object 'ap.m' not found
> ap.d <- data.frame(word = names(ap.v),freq=ap.v)
Error in data.frame(word = names(ap.v), freq = ap.v) :
object 'ap.v' not found
> table(ap.d$freq)
Error in table(ap.d$freq) : object 'ap.d' not found
> pal2 <- brewer.pal(8,"Dark2")
> png("wordcloud_packages.png", width=1280,height=800)
> wordcloud(ap.d$word,ap.d$freq, scale=c(8,.2),min.freq=3,
+ max.words=Inf, random.order=FALSE, rot.per=.15, colors=pal2)
Error in wordcloud(ap.d$word, ap.d$freq, scale = c(8, 0.2), min.freq = 3, :
object 'ap.d' not found
I checked the code again and, with R 2.15.2 (on both Windows and Linux) and a recent version of the loaded packages, everything works as expected. Two suggestions: 1) check all the packages are installed and loaded. 2) Check your internet connection and firewall settings.
RispondiEliminaHIH