I mentioned yesterday the new rating system/software launched by PLoS one. This was precipitated by an announcement on the software/technical side. Today they have an article on the PLoS blog about the new rating system, encouraging people to rate articles.
Here’s the intro from the post:
I’ve been waiting to write this Blog posting for a while and now I can. As from today PLoS ONE has a user rating system for its articles. All users can now rate articles in three subjective categories: Insight, Reliability and Style. We have made the tool, now we need you to come and use it.
User rating is a very common feature of websites these days, be it for movies, books, blog posts, pretty much anything. What user rating allows is a quick and easy survey of a communities opinion. Despite the obvious advantages to hard pressed scientists trying to get to grips with a vast literature this simple system hasn’t been much applied to scientific papers up to this point.
The major exception to this is probably Faculty of 1000, which has been providing ratings for papers for many years, but that is not based on the opinion of a whole community but only the thoughts of a select few.
So what will this new rating system look like? Well, if you go to any of the six hundred or so papers that PLoS ONE has so far published and look in the right had column you will see a little box containing five small stars. Those indicate the overall aggregate rating of the paper based on individual ‘votes’ from individual users.
What’s interesting about this is that it’s a little different from a citation index, the main way that articles are scored. You see, normally scientific articles are given a ranking or score based on how much they are cited in other articles. This is a pretty good idea, but it neglects sort of “terminal” articles — that is, articles that mark the end of most investigation into a particular niche. These articles may nonetheless be extremely interesting or useful, but never garner a large citation index. Furthermore, articles of interest to people in other fields, or even the general public, will never garner any indication of said interest or popularity under the conventional system. The occasional exception might be popular science articles inspired by new publications in Nature or Science.
With the emergence of an article rating system, that may change. People can read and rate articles without having to write an entire manuscript. People can leave comments on articles without drafting (and having accepted) an “official” editorial or response in a major academic journal. Things are getting a lot more interesting, and quickly.
As the title of the article suggests, Rate Early, Rate Often.
I find it really quite fascinating that you have got into the whole question of Open Access. Between 2002-2005, I was involved in a project to set up an Open Access institutional repository for the Rutherford Appleton, Daresbury, and Chilbolton Laboratories in the UK – here is an article on it: .
This was an implementation of the institutional repository model for Open Access publishing, where the author deposits a copy of the accepted version of their article in their local institutional repository, once the article has been accepted by a conventional journal publisher. OAI-PMH is a key technology behind this because it provides a structured mechanism for multiple institutional repositories to be ‘harvested’ by service providers who can provide cross-searching facilities.
However, I don’t think this model for Open Access has been terribly successful. There are issues with getting researchers to deposit their articles in the first place – mostly they are interested in getting their article accepted in traditional journals and cannot see what is in it for them in terms of depositing in the institutional repository. There are also issues in terms of the duplication of different versions of the article across multiple repositories, which is not ideal from a users point of view.
Anyhow, with this repository we at least made sure it met the local needs of the institution, which in this case was providing a marketing function – providing a database/record of all papers published that had benefited from the use of the laboratory facilities. So at least it was useful even if it didn’t contribute all that much to Open Access!
Personally I think a subject-orientated model is the only viable way forward, hence something like BioMedCentral or PLoS.