Review section ideas?

[Midori]

Hey there! The CAA staff would like your opinions regarding the review section's future.

I'm sure everyone knows how slowly the review section is growing. It has been very useful for providing a Christian viewpoint on anime and manga, but we have not been able to keep up with the increasing rate of anime and manga being brought to the United States. And that's to say nothing of video games. So the moderators have been discussing how we might improve our reviewing system when the next site upgrade comes around, so that it can grow further into what it was meant to be.

Some directions we're thinking of taking things are:

- Less focus on opinions and more on content. There are myriads of anime review websites that provide user-supplied reviews that are much more comprehensive than ours. So we should focus on what is unique to us, like judging the propriety of a series specifically from a Christian outlook.

- More participation from multiple users. Our current practice of approving reviews only for titles a moderator has seen is to provide a consistent and reliable viewpoint, but it is one of the reasons it's going so slowly. However, another way to provide this is to allow input from many members, so that their opinions can average out to something approaching an objective viewpoint. One possible way of doing this would be to allow all members to rate titles' various content parameters and show a final, average rating.


So we'd like to hear the community's thoughts on this subject. When it comes down to it, of course, it'll all depend on what Mithrandir and friends think they can code up, but we won't be able to do a proper job without all of you guys' opinions. Any ideas, wishes, worries, and comments are welcome. Better too many ideas than too few!

[rocklobster]

Here's an idea: I'm a reviewer on Anime Angels as well as on CAA. They have a new feature that allows you to post reviews publically as a draft. I think we should have a separate board on the forum where people can post drafts, allowing people who have seen the anime to comment on the draft so it can be improved (if need be) for the final copy, which the reviewer can then submit to the mods the normal way.

[blkmage]

Unless there's an impossibly large backlog of reviews, why don't you choose some moderators for the sole purpose of going through reviews? That way, you still have some editorial control and consistency, you have people who are on solely focused on that task without having to worry about other normal board moderating tasks, and you'd presumably be able to have people who wouldn't normally be moderators for whatever reason but would have a wider range of things they've watched or read.

[airichan623]

I second the motion! And if I can I'll help moderate if I can! ^^

I'm always wanting to add stuff to my and others' reviews.

[rocklobster]

I'd also like to help critique reviews if it comes to that.

[goldenspines]

The only downside of the idea of adding "Reviewers" is that it may become much like the same type of system we have now. It would still be one person working on one review at a time.

What I gathered from Midori's post is that a potential idea would probably be to get a majority of CAA's members involved in the reviewing process, if possible; not just a few more reviewers besides the moderators. Obviously, not every member is going to review every anime/manga (since not every member has seen every single anime/manga), but it gives a group of members an opportunity to work together to create a good and thorough review. On CAA, our members have different views on series to bring up, and this can be quite essential to making the reviews go much quicker and be more accurate overall.

This idea, of course, depends largely on formatting and how this will be set up. Therefore, it's just my two cents.

[Midori]

Indeed. Our review system currently has just one review per series, and each review is written by just one person. This has the trouble that different people have different criteria for their grading; one reviewer may find the sight of blood sickening, and so give the title a 5 in violence, while another reviewer may not mind it at all, and call it a 1 or 2. Having a review approver look over it can prevent some unreasonable grading, but it's still not very robust; it only takes into account the opinions of two people.

A lot of anime review sites allow many members to post reviews, where those reviews are freeform, like comments. Of course, this is no different than having a special forum thread for each title. To have a more formal and accessible review section we need more structure than that, and that's what we're asking you to think about.

I'm always wanting to add stuff to my and others' reviews.

This is another consideration. Our reviews that are approved already are static and never change. It'd be nice to have the opinions of later members, who were not around at the time the original review was written.

[rocklobster]

Another thing I'm wondering is how much spoilerage should we allow? Many titles change quite a lot throughout the course of the series as a result of key events. This often results in a change in tone (I'm currently watching Kemono no Souja Erin, and this certainly applies in its case. The first few episodes are kind of sweet, but after the sixth episode, things gradually get more serious).

[Peanut]

- Less focus on opinions and more on content. There are myriads of anime review websites that provide user-supplied reviews that are much more comprehensive than ours. So we should focus on what is unique to us, like judging the propriety of a series specifically from a Christian outlook.

Depending on how this is done, I might not like this. Personally, when I look at reviews (Christian or otherwise) I want to know how good the thing being reviewed is and I want that to be a significant portion of the review. With that being said, I know that the current set up doesn't really encourage detailed accounts of content. Since we're trying to have our reviews be from a Christian perspective, this doesn't really help much. I guess what I'm trying to get at is that I don't want the review section to turn into a copy of pluggedin where content is all that seems to shape the reviewers opinions.

Also, I'd like to second blkmage's suggestion. Even if it wasn't through official modship, just having some sort of group of members devoted to keeping quality control on the reviews would be sort of nice.

[ich1990]

Depending on how this is done, I might not like this. Personally, when I look at reviews (Christian or otherwise) I want to know how good the thing being reviewed is and I want that to be a significant portion of the review. With that being said, I know that the current set up doesn't really encourage detailed accounts of content. Since we're trying to have our reviews be from a Christian perspective, this doesn't really help much. I guess what I'm trying to get at is that I don't want the review section to turn into a copy of pluggedin where content is all that seems to shape the reviewers opinions.

I was under the impression that the suggestion was to turn the reviews into a "kids-in-mind" format, not a PluggedIn one. That is, avoid talking about how good it is almost entirely (there are a lot of other review sites to take care of that) and just focus listing and categorizing potentially objectionable content.

I would personally be in favor of this idea, as that is all I use the review index for anyways.

Also, giving select members the ability to review reviews and approve them sounds like a good idea, as there seems to be a very large backlog of reviews waiting to be approved. I wouldn't extend this power to everyone (have to have some sort of quality control), but I don't see why someone who has been an active part of the community for years and has 1000+ posts can't approve reviews. Heck, you could even make it so that two review approvers are required to approve a review.

[mechana2015]

I was under the impression that the suggestion was to turn the reviews into a "kids-in-mind" format, not a PluggedIn one. That is, avoid talking about how good it is almost entirely (there are a lot of other review sites to take care of that) and just focus listing and categorizing potentially objectionable content.

If we were to do that, about 50% of the shows that that the site population watches on a regular basis should just be labled 'don't bother'. Baccano and Durarara are series that were almost universally liked on this board, but in a 'kids oriented' review section it would essentially be drummed out of recommendation due to the violence level. Lets keep in mind that anime is an all ages media and CAA is an (almost) all ages forum before we go focusing our reviews on an age group that is only one portion of both the anime market and the forum membership.

[Davidizer13]

A lot of anime review sites allow many members to post reviews, where those reviews are freeform, like comments. Of course, this is no different than having a special forum thread for each title. To have a more formal and accessible review section we need more structure than that, and that's what we're asking you to think about.

Playing off of this, how about adding a more accessible way to comment on reviews? There's the forum threads for each one, but those get buried pretty quickly - we could add a link to the threads from the review itself, or we could have a more blog-style comment application embedded in the review pages.

[ShiroiHikari]

Unless there's an impossibly large backlog of reviews, why don't you choose some moderators for the sole purpose of going through reviews? That way, you still have some editorial control and consistency, you have people who are on solely focused on that task without having to worry about other normal board moderating tasks, and you'd presumably be able to have people who wouldn't normally be moderators for whatever reason but would have a wider range of things they've watched or read.

I like this idea. Maybe we could create a review "board" or "council" consisting of several members (not necessarily moderators). The only problem I foresee is the same problem that usually results from such arrangements: things never getting done because the group can't come to an agreement.

If we were to do that, about 50% of the shows that that the site population watches on a regular basis should just be labled 'don't bother'. Baccano and Durarara are series that were almost universally liked on this board, but in a 'kids oriented' review section it would essentially be drummed out of recommendation due to the violence level. Lets keep in mind that anime is an all ages media and CAA is an (almost) all ages forum before we go focusing our reviews on an age group that is only one portion of both the anime market and the forum membership.

This is also something that should be kept in mind. The vast majority of anime isn't "kid-friendly" by American standards anyway, so if it were purely focused on whether or not your five-year-old brother can watch it, every show would end up getting labeled "not recommended". I think maybe we should keep the reviewers' comments and thoughts, but maybe limit it to X number of words (if possible)?

[ich1990]

If we were to do that, about 50% of the shows that that the site population watches on a regular basis should just be labled 'don't bother'. Baccano and Durarara are series that were almost universally liked on this board, but in a 'kids oriented' review section it would essentially be drummed out of recommendation due to the violence level. Lets keep in mind that anime is an all ages media and CAA is an (almost) all ages forum before we go focusing our reviews on an age group that is only one portion of both the anime market and the forum membership.

This is also something that should be kept in mind. The vast majority of anime isn't "kid-friendly" by American standards anyway, so if it were purely focused on whether or not your five-year-old brother can watch it, every show would end up getting labeled "not recommended". I think maybe we should keep the reviewers' comments and thoughts, but maybe limit it to X number of words (if possible)?

I guess I should have made it clear that I was talking about the Kids-in-Mind site, not a kids oriented review philosophy. Take a look at one of their reviews to see what I am talking about:

http://www.kids-in-mind.com/

They have a distinctly different style than PluggedIn.

[mechana2015]

We already have those bars on our site. Those reviews very much look like plugged in, and tracking incident by incident, while logical for movies would range between difficult and suicidal depending on what anime series you're reviewing. (Try tracking all the instances of offensive content across the original Naruto or DBZ series). I would never use a review system if it looked like that, especially since many of them are chock full of spoilers.

EDIT: Regarding Kids in Mind, I looked at their Paranormal Activity 2 review and it looks exactly like a Plugged in review, except it doesn't have the reviewers opinion at all, just content information. Our reviews cover the same material, but more split up (sex and nudity are separate comments). The only variation I could see is writing longer 'offensive content summaries'.

[Peanut]

I was under the impression that the suggestion was to turn the reviews into a "kids-in-mind" format, not a PluggedIn one. That is, avoid talking about how good it is almost entirely (there are a lot of other review sites to take care of that) and just focus listing and categorizing potentially objectionable content.

I would personally be in favor of this idea, as that is all I use the review index for anyways.

Hmm, yeah I don't really like that format either though its a matter of opinion more than anything. I'm repeating myself but I do want to know whether people thought said hing being reviewed is good. Eh, but if most people want it to be that way than I guess that's fine.

[ich1990]

We already have those bars on our site. Those reviews very much look like plugged in, and tracking incident by incident, while logical for movies would range between difficult and suicidal depending on what anime series you're reviewing. (Try tracking all the instances of offensive content across the original Naruto or DBZ series). I would never use a review system if it looked like that, especially since many of them are chock full of spoilers.

Most spoilers are avoided by just making a statement non-personal. Instead of saying "the main character died by getting his head chopped off by the bad guy who was his twin brother" (which is what PluggedIn does), you would say "a character is beheaded". Not totally spoiler-less, but at least you know what the general level of violence is, and in more detail than a straight bar reading would tell (and usage of content bars isn't standardized anyway).

EDIT: Regarding Kids in Mind, I looked at their Paranormal Activity 2 review and it looks exactly like a Plugged in review, except it doesn't have the reviewers opinion at all, just content information. Our reviews cover the same material, but more split up (sex and nudity are separate comments). The only variation I could see is writing longer 'offensive content summaries'.

Pretty much, yes. Avoid specific spoilers, don't bother with personal opinion on quality, and let people know how offensive it is going to get.

For long series, the movie review format of listing everything runs into length problems. To get around this, you could give the worst it gets per category (I.E. no F-words, but other profanity), the average per category (usually only uses d*** or hell), and the average frequency (three "bad words" per episode average).

Of course, if people like to read about the quality of the series, this wouldn't help, but it does make reviews a lot easier to produce (no paragraphs to write) and group check/approve (no arguing about how good the series is or isn't). Plus, it provides the unique service of in complete "offensive content" listings, which is about the only thing not provided for by other sites.

[mechana2015]

Of course, if people like to read about the quality of the series, this wouldn't help, but it does make reviews a lot easier to produce (no paragraphs to write) and group check/approve (no arguing about how good the series is or isn't). Plus, it provides the unique service of in complete "offensive content" listings, which is about the only thing not provided for by other sites.

It was always my impression that the group check/approve was for the CONTENT not the preference side of things.

[rocklobster]

I just thought of another idea--why not PM the reviews to moderators or people who you know have seen the series? Then they could work together via PM and submit the finished work.
Also, why not a theme section? This is my bone of contention with Plugged In, many times, they refuse to take the themes of the movie into account and spend too much time nitpicking and shooting down the filmmakers. I think the theme should be taken into account for every movie. For example, Now and Then, Here and There is an extremely graphic series. However, it presents the violence as wrong and its moral is that we should not present war to children as an acceptable way to solve conflicts.

[Hiryu]

We should also have a genre section too.

Also, about the spoilerage - use general statements. I'd prefer not to have any in the review at all.

About how this anime goes along the next few episodes, "...The first few episodes are kind of sweet, but after the sixth episode, things gradually get more serious" is fine. The anime can change from good to bad from your or maybe everyone else's point of view. However saying "after the sixth episode, things gradually get more serious because the main character suffers a heart attack." is a bit on the spoilerish side.

[firestorm]

Before I suggest these suggestions I must say that...I didn't read the title post before I typed this so yeah that might be why these 1st two suggestions sound weird and why it wouldn't be any help according the stuff they asked about. XD sorry guys.

for suggestion 1: you know the page you go to when you click on someone's username? where it gives some information about them, how many posts they've made, yada yada yada? well why not have a little section in there where they can right they're fav animes they've seen and ones they want to see. like a small box in the corner you know? maybe top 5 fav ones they've seen and top like 10 they still want to see (cause I know that I always start a new series even before I get to episode 2 of one I'm watching currently though I think it's awesome XD). So that let's say people see it in their page, at least they'll know incase they want to ask that person later about it if they haven't seen it yet or if they have seen it they can start chatting over something they have in common.

I haven't really seen anything like that here unless it's a thread somewhere. In that case I'd go look for it right now. But as a feature that's embedded in your public profile page, I haven't seen it yet.


For Suggestion 2: This is for the Review page itself. Let's say I wanted to look for reviews for something that'd be suitable for the age of 16-18. I'd want to be able to put "16-18" instead of "16" and "up" like we have it currently, which would give me everything above the age range I intended for, or "18" and "down" which would give me a ton of stuff that's below the age range I intended for as well. That's alot of stuff I'd have to sift through.

Also, why not have a section called Rating right next to the Age and Name sections when your on the search page for the reviews (Note: I'm not talking about when you actually click on one of the reviews I mean the actual search page when your trying to find one). The ratings would be right there next to them and you be able to organize it by ratings, which would mean the highest rated ones could be on the top of the list if you wanted to find those. Idk I'm just giving out random Ideas at this point XD
____________________________________________________________________________________________________________________________________

To be honest I'd need time to think of something as far as a suggestion to somehow helping out with the reviewing process, but I like what everyone said so far. I just hope that if it's going down the way you guys are suggesting with people being able to comment on the reviews or what not that they don't start fighting over ratings the writers give out when they are reviewing it before it is finalized.
.... hhmmm..... don't know about the rating part but the rest yeah XD

it's 2:00 A.M. here soo...... I'm just gonna give one more suggestion which has to do with what you guys are all talking about. How about above the review have a poll about whether they liked it or not. If not they can provide a comment to improve it. If so, then you know you got a general concensus that helps with finalization of the reviews. (I know that we can create polls on threads but like someone said earlier those get buried quickly. So why not a poll on the review itself?)

Hopefully all this helped :3 Sorry for blabbering ^-^;

[rocklobster]

uh firestorm, all your suggestions are already there to begin with.

[firestorm]

uh firestorm, all your suggestions are already there to begin with.

I have corrected myself now. Go back and check it now. Thanks rock! XD

[firestorm]

I have corrected myself now. Go back and check it now. Thanks rock! XD

Have I derailed the Idea train??? :red:

[Midori]

Have I derailed the Idea train??? :red:

Heh, not really. It just coasted to a halt. But I hope this post will get it moving again.

After some discussion, the staff has an idea about how we want the review system to work. It'd be a sort of hybrid system, with each review composed of two parts.

The first part will be sliding content ratings like we have now. The difference will be that many users can submit ratings this way, with no approval process (though the moderators may be free to remove extreme submissions afterwards.) Viewers of the review will be shown an average of everyone's ratings, or some other visualization of that sort. This part will not contain any text.

The second part will be for free-form text reviews. Again, many users will be able to add to this section with no pre-approval required. It'll basically be like a forum thread, except more focused--with restrictions against off-topic posts and light conversation, for instance. This part can contain both in-depth discussion of content, and personal opinions on the quality of the show. Spoilers, of course, should be avoided or put in spoiler tags.

I think this two-part system will fit because it:
- Provides as much versatility as (or maybe more than) our current system
- Will be an order of magnitude faster, as it does not require the approval of one of a small number of members
- Will not divide the community, which could be a danger of creating a group of 'review moderators' seperate from normal users
- Is more dynamic than our current system, where reviews are unchangeable once they have been approved
- Can include the opinions of many members rather than just one member
- Can have the old reviews migrated to it without too much modification


What do you think?

[rocklobster]

I like it.

[ich1990]

The second part will be for free-form text reviews. Again, many users will be able to add to this section with no pre-approval required. It'll basically be like a forum thread, except more focused--with restrictions against off-topic posts and light conversation, for instance. This part can contain both in-depth discussion of content, and personal opinions on the quality of the show. Spoilers, of course, should be avoided or put in spoiler tags.

So this second part will be pretty much identical to the threads we currently have in the review database on each series, only more heavily moderated so that they remain on topic and useful?

Also, what will happen to the series currently within the database, will they be converted over?

[Midori]

So this second part will be pretty much identical to the threads we currently have in the review database on each series, only more heavily moderated so that they remain on topic and useful?

Yes, pretty much, except that our current review threads primarily discuss the reviews, not the series. One thing we could do to remain on topic is mandate that each post has to be at least 300 characters or something. Or we could just require moderator approval but not require the moderator to have seen the series. The other difference, of course, is that the review threads will be featured on the same page as the ratings rather than hidden away in a section of the board nobody looks at.

Also, what will happen to the series currently within the database, will they be converted over?

We can convert the ratings to be the first submission for the ratings, and the text to be the first post in the review thread. Because some of our current reviews are very high-quality, so it'd be a shame to just throw them away. The threads about the reviews, in the review board, probably will not be transferred over.

[ich1990]

Yes, pretty much, except that our current review threads primarily discuss the reviews, not the series. One thing we could do to remain on topic is mandate that each post has to be at least 300 characters or something. Or we could just require moderator approval but not require the moderator to have seen the series. The other difference, of course, is that the review threads will be featured on the same page as the ratings rather than hidden away in a section of the board nobody looks at.


We can convert the ratings to be the first submission for the ratings, and the text to be the first post in the review thread. Because some of our current reviews are very high-quality, so it'd be a shame to just throw them away. The threads about the reviews, in the review board, probably will not be transferred over.

Sounds great.

[rocklobster]

Just out of curiosity, how long would it take to set this up?