----------------------- REVIEW 1 --------------------- PAPER: 165 TITLE: Talking in the dark: An Analysis of the Anonymous Messaging Platform YikYak AUTHORS: Ramine Tinati, Dominic Difranzo, Sergej Zerr, Markus Luczak-Roesch and Wendy Hall Originality: 2 Impact: 2 Reproducibility: 3 Overall evaluation: -4 ----------- Strong Points ----------- + First work to conduct comprehensive measurement on the anonymous messaging platform YikYak. + Several interesting findings, especially for shared topics in different regions. + Paper is well written and easy to read ----------- Weak Points ----------- - Analysis made in this paper is still too surface. The authors should explore more deep insights from the collected YikYak data, like various behaviors of users over time, popularity analysis on Yaks, etc. - The motivation of analysis in network structure in the section 5.2 is not clear. - Only four regions are compared in this study, which are too coarse to see interesting findings. ----------- Detailed Review ----------- This paper conducted a comprehensive measurement analysis on the anonymous messaging platform called YikYak. The authors discussed characteristics of Yaks (posts) from various aspects including daily activities, number of reply icon identified in Yak comments, and social network structure. They also discussed topics shared by four regions in the U.S. Based on the above three points, I find the paper light in terms of novel contribution and implication of the analysis offered, I suggest the authors try to articulate why understanding of YikYak is important and how does it compare to the previous findings in the literature. ----------------------- REVIEW 2 --------------------- PAPER: 165 TITLE: Talking in the dark: An Analysis of the Anonymous Messaging Platform YikYak AUTHORS: Ramine Tinati, Dominic Difranzo, Sergej Zerr, Markus Luczak-Roesch and Wendy Hall Originality: 3 Impact: 2 Reproducibility: 2 Overall evaluation: -4 ----------- Strong Points ----------- * An interesting look at anonymous microblogging platforms across the US * The paper provides an interesting discussion session that relates YiKYak to other, more-well-studied systems ----------- Weak Points ----------- * The analysis comes across as rather cursory and does not provide much more beyond a first-level cut. * The authors derive a "network" between the campuses, but the meaning an interpretation of this network is unclear. * Many of the figures are unreadable or quite difficult to interpret. * Some methodological details are omitted, leaving the reader a bit confused. * No discussion of ethical concerns, protections, or tradeoffs. ----------- Detailed Review ----------- Overall, I like that this paper is shedding light on anonymous microblogging systems, which by their nature are difficult things to study. I found the discussion section of the paper especially fun to read. However, I have some concerns about the paper (esp. the analysis section), described below. I hope that addressing these will help to strengthen the paper. - I found the analysis section overall to come across as rather cursory. The authors present basic statistics, but do not really go into enough depth to draw any fundamental conclusions. At the end, I'm left without much of a better understanding of the student, content, users, etc of YikYak than before. Given the data the authors are working with (i.e., the lack of user identifiers), it seems that the primary thing that would be interesting to focus on are the topics -- I was expecting the authors to go into a deep dive here (applying LDA to the entire thing, examining how different topics have different discussion patters, time-of-day effects, relating topics to external events, etc), but not much of this is presented. I'd encourage the authors to try and dig a little deeper to see if they can provide a stronger grasp of YikYak. - I completely did not follow why examining the "network" of mentions between campuses is a reasonable thing to do. The authors give the example of @starbucks, but what exactly does it mean to link together campuses where @starbucks is mentioned? I simply don't know what to make of this, and the authors do not provide much guidance. - Many of the figures are either unreadable or are quite difficult to interpret. Figures 5, 6 and 7 are examples, with each of these having unclear take-aways. I feel these could be better presented by either showing less data (e.g., focusing on only certain aspects) or by statistics instead. - Some of the methodological details are omitted -- for example, how were the authors able to actually access this data? Is there any bias in their data collection methodology? Did they capture all "Yaks"? All campuses? Etc. - Finally, there is no discussion of research ethics in the paper. The authors are dealing with very sensitive data that users are intending to only share with local users anonymously. However, the authors are collecting it all up and using it for research purposes. What protections were put into place? What happens if the authors ran across people's names or other personal information? To be clear, I don't think the authors acted unethically, but, in a paper that deals with this kind of sensitive information, I would expect an explicit discussion of the nature of the data, the ethical concerns, the tradeoff between harm and benefits, and the protections the authors took to minimize harm to the users whose data they examined. - They removed any Yak which not not appear to have a valid location, so are there other data cleaning techniques applied, for duplicates or empty yaks? How about the ones that contain only emoticons or other special characters? - I have some confusion about the datasets as well, with a total of 515,704 Yaks collected in 12 days from 9,705 US campuses, that means they collect an average of 4.4 Yaks per day per campus, which is a really small size. So they should add the Max/Min/Med/Ave # of Yaks per campus in the Table 1. Also Yik Yak official website claims over 2,000 US campuses, so I’m not sure whether the number 9,705 in the paper is correct. And I think they could add some figures to make it more clear: 1. CDF figure of the # of Yaks per campus 2. CDF figure of the total # of comments per Yak - Also, a previous study in [27] have over 800,000 Yaks collected from just a single location over 5 months. It's not clear why studying 9,705x more locations (for 1/10th the time) got so few yaks. - In 2nd paragraph of the Section 5.1, they mentioned that there is no limitation to the length (characters) of a Yak, and this is wrong, because Yik Yak states clearly on their website that Yak has a limit of 200 characters (no spaces, emojis welcomed). See link: https://yikyak.zendesk.com/hc/en-us/articles/211749926-What-s-the-difference-between-Now-and-a-yak- - A missing reference is “Northcut, K.M., "Dark side or insight? Yik Yak and culture on campus," IPCC, 2015”, where the authors defined categories and included stigmatized topics such as racism, vulgarity, discrimination, emotional expression, and others. While it is not necessary to apply the same methodology in other studies, similar analysis can significantly improve this section. - The paper is easy to read, but there are some grammar errors, or typos, or inconsistence. Some examples are below: + last paragraph in Section 2: “We also will contributes to the …” => “contribute to …” + 1st paragraph in Section 5: “topic modelling” -> “topic modeling” + 3rd paragraph in Section 6: “widespread use of the “1@” mention …” => “of the “@” mention” + 2nd paragraph in Section 7: “is how individuals’ are using …” => “how individulas are …” + The usage of some terms in this paper are not consistent, such as they use Reply Icons in 3rd paragraph of Section 6, but reply icons in the following sentence. + and more … ----------------------- REVIEW 3 --------------------- PAPER: 165 TITLE: Talking in the dark: An Analysis of the Anonymous Messaging Platform YikYak AUTHORS: Ramine Tinati, Dominic Difranzo, Sergej Zerr, Markus Luczak-Roesch and Wendy Hall Originality: 1 Impact: 1 Reproducibility: 2 Overall evaluation: -4 ----------- Strong Points ----------- the platform the authors set to analyze has some interesting features related to anonymity ----------- Weak Points ----------- none of the results presented is novel, and no insights are obtained on the most prominent platform's characteristic (anonymity) ----------- Detailed Review ----------- The paper explores the structural characteristics of the anonymous social platform yikyak. The authors collected half million yaks from about 10 thousand US locations matching campuses. They use LDA for topic modeling to identify the topical structure of the conversation. I deem that the authors could have used the space available in a much more effective way: the actual content is only about 5.5 pages, with Fig.s 1 and 2 that are not very informative, and Fig.s 5-6-7 occupy a full page but look like "spaghetti monsters", very hard to interpret or even read. Equivalently the fonts of fig. 3 and 4 are impossible to read. The results are generic, and do not yield any peculiar insights on one of the most important aspects of this study that would have been the role of anonymous communication. By admission of the authors themselves, several other studies already highlighted similar or equivalent structural results on other platforms. As a summary, the contribution is very incremental and not up to the WWW requirements. ----------------------- REVIEW 4 --------------------- PAPER: 165 TITLE: Talking in the dark: An Analysis of the Anonymous Messaging Platform YikYak AUTHORS: Ramine Tinati, Dominic Difranzo, Sergej Zerr, Markus Luczak-Roesch and Wendy Hall Originality: 2 Impact: 2 Reproducibility: 3 Overall evaluation: -2 ----------- Strong Points ----------- --Analyzes a relatively new class of anonymous messaging services --Data spans across a full country --Interesting results about the content of posts and geographic variation ----------- Weak Points ----------- --A lot of descriptive analysis, but many results are in line with expectation based on past studies in social media --The paper fails to motivate "why is this important", or select a key research question to tackle. In the absence of it, reads like a collection of results about YikYak. --Discussion could have had better context with comparison with non-anonymous services and related findings therein ----------- Detailed Review ----------- This paper describes patterns of user activity on YikYak, an anonymous message posting service. Some of the results presented, such as the geographic patterns of word usage, are interesting, but unfortunately, as with many descriptive papers, it is hard to pinpoint the key contribution of the paper. I can think two ways of enhancing the contribution: 1. It might be good to lead with a key question: e.g. how anonymity makes yikyak different and organize the paper around the findings. Would also be good to compare past studies on other social media in this setup. 2. If the authors choose to keep the descriptive theme of the paper, then it can be good to think of more high-level and derived descriptives. E.g. the authors suggest that Yikyak is a mixture of many hyperlocal communities. A connected descriptive analysis would be to see the diversity in messages and compare that with distances from campuses. Instead of the hairball networks in Figures 5 and 6 (which look great but do not convey much information), it will be good to present some analysis on diversity as it pans out with distance, both geographically and lexicographically. Another idea could be within campus versus between campuses diversity. These are just two ideas, the big picture is that having descriptives that connect to higher-level constructs or research questions will help place this paper in the broader understanding of social media. ------------------------- METAREVIEW ------------------------ There is no metareview for this paper