Mining Tweets for Tag Recommendation on Social Media
Denzil Correa and Ashish Sureka
Indraprastha Institute of Information Technology
(IIIT-Delhi), India
{denzilc, ashish} @iiitd.ac.in
http://www.iiitd.ac.in/
ABSTRACT
Automatic tag recommendation or annotation can help in
improving the eciency of text-based information retrieval
on online social media services like Blogger, Last.FM, Flickr
and YouTube. In this work, we investigate alternate solu-
tions for tag recommendations by employing a Wisdom of
Crowd approach in a mashup framework. In particular, we
mine tweets on Twitter and use their hashtag(s) and con-
tent to annotate videos on Flickr, Photobucket, YouTube,
Dailymotion and SoundCloud. We crawl Twitter to col-
lect a random sample of tweets containing Flickr, Photo-
bucket, YouTube, Dailymotion and SoundCloud URLs. We
then recommend tags for these services using hashtag(s) and
content present in tweets. We use a hybrid technique (auto-
mated and manual) to validate our results on dierent sub-
sets (presence / absence of hashtags, presence / absence of
media tags) of data. Experimental results demonstrate that
the proposed solution approach is eective and reliable.
Categories and Subject Descriptors
H.3 [Information Storage and Retrieval]: Information
Search and Retrieval|Information Filtering
General Terms
Algorithms, Design, Experimentation, Measurement, Veri-
cation
Keywords
Online Social Media, Web 2.0, Tag recommendation, Twit-
ter, Crowdsourcing, Mining user generated content
1. RESEARCH MOTIVATION AND AIM
Several popular social media websites and Web 2.0 plat-
forms allow web-users to supply tags (also called as labels,
keywords, textual annotations) describing the content of the
web-source (documents, photo, songs, videos, web-links) to
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
SMUC’11, October 28, 2011, Glasgow, Scotland, UK.
Copyright 2011 ACM 978-1-4503-0949-3/11/10 ...$10.00.
provide contextual information. Some popular examples of
such Web 2.0 services are: CiteULike (free online service
for managing and discovering scholarly references), Delicious
(social bookmarking for online articles), Flickr (online photo
management and sharing application), Last.FM (online mu-
sic recommendation and sharing), WordPress (weblogs) and
YouTube (online video sharing). These services use tags to
facilitate search [1], information retrieval [4] [3], object clas-
sication [6], content discovery and exploration [12].
Garg et al. report that most of the time users add very
few tags or even none at all. Their study on Flickr photo
sharing system indicates that at least 20% of public photos
have no tag at all [2]. Sigurbjornsson and van Zwol report
that 64% of the photos have 1-3 tags (a study consisting of
over 52 million publicly available Flickr photos) [8]. Song
et al. perform a study on tagging on CiteULike and report
that the average number of tags per paper was 3.35 (a study
consisting of 32242 entries, with 9623 distinct papers) [9].
Zhuang et al. reveal that web images are usually not an-
notated with proper tags, several of them are completely
unlabeled and tags are often incomplete for describing the
content of the images [14]. Some previous studies show that
several tags in Flickr are imprecise and around 50% of the
tags are unrelated to the image content [4][5]. Moxley et al.
reveal that user-generated annotations on web video repos-
itories such as YouTube are not quality-controlled (annota-
tions are typically incomplete and noisy, incorrect keywords,
missing quite a few relevant ones) [6]. Toderici et al. present
a system that automatically recommends tags for YouTube
videos whose titles are too short or whose descriptions and
tags are either brief or missing (calling the phenomenon as
metadata deserts) [12].
These research ndings clearly indicate major quality is-
sues with free-form user-provided web-resource tags on var-
ious social media Web 2.0 platforms. The quality of user-
supplied web-resource tags (precision, completeness, quan-
tity, relevance, richness) have a profound impact on the ef-
cacy of web-search, navigation, browsing and recommen-
dation systems exploiting these tags for decision making.
In order to address the tag quality problem in online social
media services (such as CiteULike, Delicious, Flickr, Word-
Press, YouTube), several techniques have been proposed for
automatic tag recommendation and annotation (the focus
of this work) for user-generated web-resources (images, we-
blogs, songs and videos). In this work, we propose a Wisdom
of Crowd mashup solution framework for tag recommenda-
tion on online social media services. We use Twitter to rec-

ommend tags for Flickr1, Photobucket2 (image hosting and
sharing), YouTube3, Dailymotion4 (video hosting and shar-
ing) and SoundCloud5(audio hosting and sharing).
Twitter (dened as real-time information network6) is a
very popular and fast growing micro-blogging service which
allows users to post stream of messages of up-to 140 charac-
ters in length (called tweets) and follow other users (called
Twitterers). Twitter has attracted worldwide popularity.
As of June 2011, on an average there are 200 million tweets
posted per day7 and an average 460,000 new accounts were
created per day during the month of February 20118. Twit-
terers also use various external social media services like
Flickr and Photobucket, Youtube and Dailymotion, Sound-
Cloud in order to share photographs, videos and songs.
The research motivation of this work is to investigate al-
ternate and eective automated solutions to semantically
tag a phenomenal amount of inadequately annotated user-
generated web-resources and address the tag quality prob-
lem. The specic research aim of this paper is to investigate
the application of mining tweets to semantically enrich the
tags of popular image(Flickr, Photobucket), video(YouTube,
Dailymotion) and songs(SoundCloud) sharing services.
The rest of the paper is organized as follows : Section 2
gives a brief overview of the related work and contribution,
Section 3 states the research questions, Section 4 describes
our methodology, Section 5 explains the evaluation tech-
niques and metrics used in our study, Section 6 shows the re-
sults on the evaluation metrics, Section 7 outlines some dis-
cussion and Section 8 draws the conclusion from our study.
2. RELATEDWORKANDCONTRIBUTIONS
Previous studies describe techniques for automatic tag
annotation[2][8][9][10], classication[7], clustering[1] and re-
nement [5][14] in various social media services like bib-
sonomy, online bookmarks, images, songs and videos. We
compare and contrast our work with the three most closely
related studies.
Overell et al. present a generic method for classifying
tags into semantic categories using third party open con-
tent resources, such as Wikipedia and the Open Directory
apply their system for classifying Flickr tags [7]. The sim-
ilarity between their study and our work is the application
of a third party resource for classifying tags in a photo shar-
ing website. Zhao et al. present a method to annotate web
videos by performing experiments on cross sources (annotat-
ing Google and Yahoo! videos using YouTube videos) [13].
The similarity between their study and our work is the ap-
plication of using cross-sources for video tagging. Stewart et
al. present a method for cross-tagging where tags from one
social system are recommended in order to automatically
annotate resources in another social system. They perform
experiments on Blogger (blogging site) and Last.fm (a social
music site) [11]. The similarity between their study and our
1http://www.flickr.com/
2http://photobucket.com/
3http://youtube.com/
4http://www.dailymotion.com/
5http://soundcloud.com/
6http://twitter.com/about/
7http://blog.twitter.com/2011/06/
200-million-tweets-per-day.html
8http://blog.twitter.com/2011/03/numbers.html
work is in the application of the concept of cross-tagging
across social media platforms.
In comparison to previous related work there are several
key dierences between previous work and our study. This
study looks at the problem of automatic tag annotation from
a dierent and fresh perspective. In context to previous
research, this paper makes the following unique contribu-
tion. This is the rst study on automatic tag recommen-
dations for Flickr, Photobucket, YouTube, Dailymotion and
SoundCloud using Twitter. The proposed method consists
of mining tweets to semantically enrich web-resources on
image, video and audio sharing websites. We perform ex-
periments on data crawled from Twitter and Flickr, Photo-
bucket, Dailymotion, YouTube, SoundCloud. Experimental
results demonstrate that a Wisdom of Crowd mashup frame-
work can be an eective solution for tag recommendation.
3. RESEARCH QUESTIONS
Twitter denes hashtags as keywords or topic marked by
a user in a tweet 9. Hashtags are preceded by the # symbol
and are used to categorize the tweet. Hashtags can occur
anywhere within the message body and this feature enables
convenient searching as users can click on a hashtag in the
tweet to retrieve other tweets in that category. We man-
ually inspected tweets containing URLs to external social
media services (like Flickr, YouTube etc.). We notice that
both hashtags and the tweet content can be used to label
the associated multimedia object. Table 1 shows illustrative
examples to explain our intuition. Figure 1 shows snapshots
of photographs to examples in Table 1. Examples I and III
show how hashtags in tweets can be used for tag enrich-
ment while Examples II and IV demonstrate that content in
tweets can be used for tag recommendation in the absence
of tags in the associated multimedia objects. Based on our
intuition we formulate and address the following research
questions :
 Research Question 1(RQ1): Can hashtags in tweets
be exploited to automatically tag external media ob-
jects in Flickr, YouTube etc.?
 Research Question 2(RQ2): Can hashtags with
content in tweets be exploited to automatically tag
external media objects in Flickr, YouTube etc.?
4. METHODOLOGY
In this section, we describe our solution approach and data
collection.
4.1 Solution Approach
Figure 2 details our proposed solution framework. The so-
lution approach primarily consists of two concurrent phases
{ Phase A and Phase B. The output from both these
phases are then passed to the Evaluation Module to ob-
tain the results.
4.1.1 Phase A
The tweets from the datastore are passed to the Filter-
ing Module. This modules rst tokenizes each tweet us-
9http://support.twitter.com/entries/
49309-what-are-hashtags-symbols

Table 1: Illustrative examples provided as evidences to show that the hashtags and content in tweets have
potential to semantically enhance multimedia retrieval
Serial
No.
Tweet Hash Tags Service Media Media Tags
I Chimp Photo of the Week:
http://
ic.kr/p/ab9NQy #chimp
#chimp Flickr JGI, Jane Goodall, Jane
Goodall Institute, chimp,
chimpanzee, Tchimpounga,
sanctuary
II Lovely Heart Curves [#VisualArt
#Graphics #Illustration #Math #Curves
#Shape] http://pbckt.com/p5.UEB3Av
#VisualArt #Graph-
ics #Illustration
#Math #Curves
#Shape
Photobucket visual art, math curves,
graphics, illustration
III Stunning and delicate neck-
lace in #silver sterling #ligree
http://pbckt.com/pL.dFmmux
#silver #ligree Photobucket No Tags
IV The grave of General Anders at
the Polish War Cemetery, Monte
Cassino #ww2 http://
ic.kr/p/9LgBSq
#ww2 Flickr No Tags
Figure 1: Snapshot of photos associated with tweets to illustrate motivating examples in Table 1
Figure 2: Proposed solution framework for automatic tag recommendation on Flickr, Photobucket, YouTube,
Dailymotion, SoundCloud by exploiting tweets

ing a WhiteSpaceTokenizer10. Next, all tokens containing
URLs, stopwords11 and @-mentions(which indicate replies
to a Twitterer) are ltered out. The above mentioned to-
kens wouldn't enhance or enrich multimedia and hence, we
lter out such low quality content. We then segregate tokens
from tweets which contain hashtag(s) from the ones without
hashtag(s). We call these tokens TTH and TTNH respec-
tively.
4.1.2 Phase B
The tweets from the datastore are passed to the URL
Parser. The parser lters out the URL and passes the URL
to the Media Web 2.0 API module. This module calls the
respective Web 2.0 API (Flickr 12, Photobucket 13, YouTube
14, Dailymotion 15, SoundCloud 16) and collects the informa-
tion from the media URL. We then segregate media objects
which contain tag(s) from ones without tag(s). We call these
MT and MNT respectively.
4.1.3 Evaluation Module
This module takes the input TTH, TTNH and MT, MNT
and outputs scores on various evaluation metrics. Section 5
details the evaluation methodology and metrics used.
4.2 Data Collection
We use the Twitter Search API 17 to collect relevant tweets
for our experiments. Table 2 shows the various terms used
to query the Twitter Search API. We tried dierent varia-
tions of queries and selected the ones which gave the most
unadulterated tweets in the results. For example, tweets
starting with RT signify Retweet (similar to forward in e-
mail). These can essentially be treated as duplicates and
may not semantically enhance the associated multimedia ob-
ject. Hence, we lter such tweets from our results. Due to
limitations of the Twitter Search API we are able to ob-
tain 7500 tweets for each social media service. We lter out
non-English tweets by using the iso language code parame-
ter option in tweets as returned by the API. Our approach
is language independent, however, in this work we only per-
form experiments on English tweets. We then lter out du-
plicate tweets by performing a simple string match. The
number of tweets for each media service is given in Table 3.
For each social media service, we can divide the total num-
ber of tweets into four categories based on the presence or
absence of hashtags in tweet and presence or absence of tags
in the associated media object. Table 4 details the number
of tweets present in various categories.
5. EVALUATION
This section discusses the validation technique and evalu-
ation metrics used in our experiments.
10http://nltk.googlecode.com/svn/trunk/doc/api/
nltk.tokenize.regexp.WhitespaceTokenizer-class.
html
11http://nltk.googlecode.com/svn/trunk/doc/api/
nltk.corpus-module.html#stopwords
12http://www.flickr.com/services/api/
13http://photobucket.com/developer
14http://code.google.com/apis/youtube/1.0/
developers_guide_python.html
15http://www.dailymotion.com/doc/api/rest-api.html
16http://developers.soundcloud.com/
17https://dev.twitter.com/docs/using-search
Table 2: Search terms used to query Twitter for
dierent social media services
Service Search Terms
Flickr
ic.kr -RT
YouTube youtu.be -RT -uploaded
-liked -favorited
Photobucket pbckt.com -RT
DailyMotion dai.ly -RT
SoundCloud snd.sc -RT
Table 3: Dataset size for dierent social media ser-
vices
Service Number of Tweets
Flickr 2833
YouTube 2280
Photobucket 1487
DailyMotion 290
SoundCloud 5796
5.1 Validation Technique
In order to perform validation on our dataset of tweets, as
described in Table 4, we see two possible cases. Tweets : (i)
with tags in associated media objects and (ii) without tags
in associated media objects. For (i), we use an automated
technique while for (ii) we use a manual technique. We now
explain these techniques and motivation behind using them.
5.1.1 Automated Technique
One of the methods to measure the ecacy of an auto-
mated tag recommendation system is to compare the out-
put of an automated solution against dataset which is pre-
annotated or pre-tagged (a test dataset containing the ground-
truth, web-resources tagged by the owner or the user who
uploaded the object on the social media). Song et al. use
evaluation metrics such as top-k accuracy, exact-k accuracy,
tag-recall and tag-precision and assess the performance of
their technique based on presence of tags recommended by
their algorithm in the tag-set annotated by the user [9].
There are some limitations of this validation approach as
a tag recommended by an automated solution not being
present in the pre-annotated dataset does not mean that
the tag is not relevant. Previous studies report that the av-
erage number of tags on web-resources is low (between 0-4)
[2][8][9]. This naturally decreases the probability of a match
between the predicted output and actual output for such re-
sources. Also, due to presence of synonymy and morphologi-
cal variations in natural language, a concept match may not
mean a match in terms. However, we believe that the per-
centage of predicted tags by our technique being present in
the pre-annotated tag-set (by the web-user) is a reasonable
indicator on the quality of the automatic tag recommenda-
tion technique.
In this technique, we perform a simple string match by
tokens obtained from tweets with tags obtained from the as-
sociated media (MT). We observe that a lot of tweets in our
dataset are auto-generated due to user accounts from exter-
nal social media services being linked (sharing of Activity
Feed) to Twitter. Such tweets generate a common hash-
tag like #Snapbucket for Photobucket and #SoundCloud

Table 4: Tweets belonging to dierent buckets depending on the presence or absence of hashtag and presence
or absence of tags associated with media objects
Tweets
Media Type Service Media With Hashtags Without Hashtags
Video
YouTube
with Tags 656 1566
without Tags 14 44
Dailymotion
with Tags 65 196
without Tags 5 24
Image
Flickr
with Tags 613 1470
without Tags 160 590
Photobucket
with Tags 24 569
without Tags 92 802
Music SoundCloud
with Tags 189 2944
without Tags 118 2545
for SoundCloud. We do not consider such hashtag(s) from
the tweets while performing the validation.
5.1.2 Manual Technique
As discussed by Garg et al., one of the ways of evaluat-
ing the ecacy of a tag recommendation system is manual
validation by users or human annotators [2]. While manual
validation of system output by users has been a scienti-
cally acceptable form of performance evaluation methodol-
ogy, it is not perfect due to inherent subjectivity, missing
context, human errors, dierent guidelines, dierent senses,
disagreements between annotators and ambiguities. Garg
et al. mention that what is considered relevant tag to a
given picture (and just the picture without any context or
background information) can vary from user to user due to
dierence in perception [2]. We agree that validation by
few users is not perfect but we also believe that accuracy
results based on manual validation can still provide useful
insight into the quality of tag recommendation techniques.
Hence, user validation can aid evaluation of automatic tag
recommendation systems.
In this technique, we ask annotators to check if tags rec-
ommended by our system semantically enhance the informa-
tion in the associated multimedia object. A tag semantically
enhances the associated media object if it provides more in-
formation than currently present tags, title and description
in the media object. Figure 3 shows some examples describ-
ing positive and negative cases of semantic enhancement.
Figure 3: Snapshot of photos associated with tweets
to illustrate semantic enhancement of images using
tweet content
5.2 Evaluation Metrics
In order to formalize the evaluation metrics { Overlap
Count, Average Overlap Percentage and Relevance Score
we rst dene the following.
HTi = Set of Hashtags in Tweeti
Ti = Set of media tags corresponding to Tweeti
TTi = Set of tokens in Tweeti
N = Total number of Tweets
We now dene evaluation metrics for each of the two val-
idation techniques viz. automated and manual.
5.2.1 Automated
We use two metrics to evaluate our tag recommendation
system { Overlap Count (OC) and Average Overlap Per-
centage (AOP). We dene them as follows:
Overlap Count (OC).
Overlap Count measures the number of overlaps of tags
recommended by our system with the tags present in the
associated media object.
OC = Ni=1jHTi \ Tij
Average Overlap Percentage (AOP).
In order to calculate Average Overlap Percentage, we rst
calculate OC of each tweet per number of tokens in that
tweet (TTi). We then take percentage average across all
tweets.
AOP = ( Ni=1

jHTi \ Tij
jTTij

=N )  100
5.2.2 Manual
In the manual validation technique, we use Relevance Score
(RS) to evaluate our tag recommendation system. We dene
them as follows:
Relevance Score (RS).
Relevance Score measures the average number of relevant
tags by our tag recommendation system with respect to the
associated multimedia object.

Table 5: Experimental Dataset (X= f Y: YouTube, D : Dailymotion, F : Flickr, P: Photobucket, S : Sound-
Cloudg , DXi denotes number of tweets containing URLs of X to answer RQi, 8i = f1; 2g)
Validation Technique
Service Media No. of tweets Automated Manual
YouTube
DY1 656 14
DY2 656 + 1566 = 2222 14 + 44 = 58
Dailymotion
DD1 65 5
DD2 65 + 196 = 261 5 + 24 = 29
Flickr
DF1 613 160
DF2 613 + 1470 = 2083 160 + 590 = 750
Photobucket
DP1 24 92
DP2 24 + 569 = 593 92 + 802 = 894
SoundCloud
DS1 189 118
DS2 189 + 2944 = 3133 118 + 2545 = 2663
Ri =
8
<
:
1 if content of Tweeti is marked
relevant by annotators
0 otherwise
RS =
Ni=1Ri
N
6. RESULTS
In order to address the research questions given in Sec-
tion 3 with appropriate validation techniques (automated
and manual), we divide our dataset into four buckets for
each media service. Table 5 describes the number of tweets
present in each bucket for each media service and corre-
sponding validation technique. We use the following four
cases to detail the same:
* Case I : RQ1-Automated In this case, we consider
those tweets which contain hashtags and the corre-
sponding media also contains tags. In Table 5, fDY1,
DD1, DF1, DP1, DS1g{Automated cells represent the
number of tweets corresponding to this case. Exam-
ple I of Table 1 illustrates this case. We perform au-
tomated validation as discussed in Section 5.1.1. We
report our results on the overlap count and average
overlap score metrics for this case.
* Case II : RQ2-Automated In this case, we consider
those tweets which may or may not contain hashtags
but the corresponding media contains tags. In Ta-
ble 5, fDY2, DD2, DF2, DP2, DS2g{Automated cells
represent the number of tweets corresponding to this
case. Example II of Table 1 illustrates this case. We
perform automated validation as discussed in Section
5.1.1. We report our results on the overlap count and
average overlap score metrics for this case.
* Case III: RQ1-Manual In this case, we consider
those tweets which contain hashtags but the corre-
sponding media doesn't contain tags. In Table 5, fDY1,
DD1, DF1, DP1, DS1g{Manual cells represent the num-
ber of tweets corresponding to this case. Example III
of Table 1 illustrates this case. We perform manual
validation as discussed in Section 5.1.2. We report our
results on the relevance score metric for this case.
* Case IV: RQ2-Manual In this case, we consider
those tweets which may or may not contain contain
hashtags and the corresponding media doesn't contain
tags. In Table 5, fDY2, DD2, DF2, DP2, DS2g{
Manual cells represent the number of tweets corre-
sponding to this case. Example IV of Table 1 illus-
trates this case. We perform manual validation as dis-
cussed in Section 5.1.2. We report our results on the
relevance score metric for this case.
Table 6 shows results on evaluation metrics (OC, AOP)
for the automated validation technique and Table 7 shows
results on evaluation metric (RS) for the manual validation
technique.
The Overlap Count and Average Overlap Scores in Table
6 suggest that there's a high amount of overlap between
tweet content (with or without hashtags) and the associated
media tags. The Relevance Scores in Table 7 demonstrate
that tweet content (with or without hashtags) can be used
to semantically annotate and enrich associated multimedia
objects.
7. DISCUSSION
Our solution employs a Wisdom of Crowd mashup frame-
work for social tag recommendation. Our solution is lan-
guage independent and hence, can be used to recommend
multi-lingual tags on social media services. Our tag rec-
ommendation system could be used to enhance Information
Retrieval systems like news recommendation, online shop-
ping and personalization of online content. However, a more
sophisticated algorithm to selectively choose content from
tweets as tags could be used to improve the quality of rec-
ommended tags. Also, the coverage our system would be
high for multimedia objects only on external social media
services which are frequently shared on Twitter. Hence, our
technique is a complementary approach to the existing work
on automatic tag recommendation systems.
8. CONCLUSION
We present an approach to automatically recommend tags
on social media services. The proposed approach employs a
crowdsourcing framework by mining Twitter to recommend
tags on image, video and audio sharing services like Flickr,
Photobucket, Dailymotion, YouTube and SoundCloud. We
use a hybrid technique to perform validation and propose

Table 6: Evaluation Metrics for Automated Validation Technique (OC : Overlap Count, AOP : Average
Overlap Percentage)
Evaluation Metric
Service Media Overlap Count (OC) / Number of Tweets Average Overlap Percentage (AOP)
YouTube
RQ1 133/656 1.92
RQ2 1883/2222 5.17
Dailymotion
RQ1 12/65 1.88
RQ2 360/261 17.71
Flickr
RQ1 420/613 7.78
RQ2 1580/2083 5.58
Photobucket
RQ1 17/24 9.45
RQ2 530/593 17.14
SoundCloud
RQ1 18/189 0.89
RQ2 4508/3133 15.41
Table 7: Evaluation Metrics for Manual Validation Technique
Relevance Score
YouTube Dailymotion Flickr Photobucket SoundCloud
RQ1 85.71% 40 % 79 % 40.91 % 57.84 %
RQ2 84.48% 29.62% 53.86% 37.02 % 62.59 %
appropriate evaluation metrics corresponding to the tech-
niques. The results show that content of tweets can be used
to recommend tags on various social media services.
Acknowledgments
The authors would like to thank Anupama Aggarwal for
annotations and feedback on various drafts of this paper.
We would also like to thank the Department of Informa-
tion Technology (DIT), India and the National Internet Ex-
change of India (NIXI) for their support.
9. REFERENCES
[1] G. Begelman, P. Keller, and F. Smadja. Automated
tag clustering: Improving search and exploration in
the tag space. In Collaborative Web Tagging Workshop
at WWW2006, Edinburgh, Scotland, 2006.
[2] N. Garg and I. Weber. Personalized, interactive tag
recommendation for
ickr. In ACM conference on
Recommender systems, RecSys '08, pages 67{74, New
York, NY, USA, 2008. ACM.
[3] I. Katakis, G. Tsoumakas, and I. Vlahavas. Multilabel
text classication for automated tag suggestion. In
ECML/PKDD 2008 Discovery Challenge, 2008.
[4] L. S. Kennedy, S.-F. Chang, and I. V. Kozintsev. To
search or to label?: predicting the performance of
search-based automatic image classiers. In MIR, MIR
'06, pages 249{258, New York, NY, USA, 2006. ACM.
[5] D. Liu, X.-S. Hua, L. Yang, M. Wang, and H.-J.
Zhang. Tag ranking. In WWW, WWW '09, pages
351{360, New York, NY, USA, 2009. ACM.
[6] E. Moxley, T. Mei, and B. S. Manjunath. Video
annotation through search and graph reinforcement
mining. IEEE Transactions on Multimedia,
12(3):184{193, Apr 2010.
[7] S. Overell, B. Sigurbjornsson, and R. van Zwol.
Classifying tags using open content resources. In
WSDM, WSDM '09, pages 64{73, New York, NY,
USA, 2009. ACM.
[8] B. Sigurbjornsson and R. van Zwol. Flickr tag
recommendation based on collective knowledge. In
WWW, pages 327{336, New York, NY, USA, 2008.
ACM.
[9] Y. Song, Z. Zhuang, H. Li, Q. Zhao, J. Li, W.-C. Lee,
and C. L. Giles. Real-time automatic tag
recommendation. In SIGIR, SIGIR '08, pages
515{522, New York, NY, USA, 2008. ACM.
[10] S. C. Sood, S. H. Owsley, K. J. Hammond, and
L. Birnbaum. Tagassist: Automatic tag suggestion for
blog posts. 2007.
[11] A. Stewart, E. Diaz-Aviles, W. Nejdl, L. B. Marinho,
A. Nanopoulos, and S.-T. Lars. Cross-tagging for
personalized open social networking. In ACM
conference on Hypertext and hypermedia, HT '09,
pages 271{278, New York, NY, USA, 2009. ACM.
[12] G. Toderici, H. Aradhye, M. Pasca, L. Sbaiz, and
J. Yagnik. Finding Meaning on YouTube: Tag
recommendation and Category Discovery. In
CVPR'10, 2010.
[13] W.-L. Zhao, X. Wu, and C.-W. Ngo. On the
annotation of web videos by ecient near-duplicate
search. IEEE Transactions on Multimedia,
12(5):448{461, 2010.
[14] J. Zhuang and S. C. Hoi. A two-view learning
approach for image tag ranking. In WSDM, WSDM
'11, pages 625{634, New York, NY, USA, 2011. ACM.