Clarify ---

Each participant of "the panel" submits their valuation *WITH NO KNOWLEDGE* of the valuation made by others -- This means each valuation is independent (honest?). You then throw out the highest and lowest valuations. You can also use fairly simple math to detect if members of the panel are actually talking to each other when they should'nt be. ;-)

If the identity of the valuations are to be withheld that's probably a good thing to. But there should be a "warm body" who presents the data, does not sit on the panel, and *DOES* know who made each valuation.

I just realized that there is an aspect of "hidden identity" that I had not considered so let me adjust my comment --

Each panel member could have a unique avatar seperate from their DomainState avatar and this unique avatar *IS* published with their valuations. Thus everyone can get a sense of how each panel member votes without knowing who the person is.

I'm not advocating hiding the panels identity, but I am advocating having the panel members submit their valuations without knowledge of the other valuations.

It might be technically difficult (and therefore comes with a lot of work on the part of the Administrators) if not impossible to set up a complicated appraisal panel system that carries your suggestions, ILikeInfo. And I didn't expect this idea that I proposed earlier on would turn out to be some kind of "star chamber".

i posted this at the same time as NameSlave before I saw his post... please pardon some repetition

Originally posted by ILikeInfo
Clarify ---

Each participant of "the panel" submits their valuation *WITH NO KNOWLEDGE* of the valuation made by others -- This means each valuation is independent (honest?). You then throw out the highest and lowest valuations. You can also use fairly simple math to detect if members of the panel are actually talking to each other when they should'nt be. ;-)

If the identity of the valuations are to be withheld that's probably a good thing to. But there should be a "warm body" who presents the data, does not sit on the panel, and *DOES* know who made each valuation.

I can see the benefits of people not knowing what each other are thinking before they vote. I was, however, originally thinking of something which would be relatively simple using the existing thread/poll software to collect data.... basically a minor variation on the current appraisal/polling process... in which people can see previous valuations. Would the the above would require manual compiling of PMs or special programming? (my preference to avoid)

I do appreciate the ideas!

the current system would mean each person would need to setup a poll for their domain as polls are anonymous to viewers and then the poll details would need to be collated and maybe a timeframe to close the poll imposed.

But, if theres a better way for the system to work then we could look at the mechanics of it to reduce admin and make it more relevant to the desired outcome.

Originally posted by safesys
the current system would mean each person would need to setup a poll for their domain as polls are anonymous to viewers and then the poll details would need to be collated and maybe a timeframe to close the poll imposed.

But, if theres a better way for the system to work then we could look at the mechanics of it to reduce admin and make it more relevant to the desired outcome.

Very gracious of you Safesys considering all the time you are already putting in

My current inclination is that minimum number of votes for inclusion in list (10? 20?) and use of median should greatly reduce abuse by shill teams and also maintain openness so that all can participate...

..but this discussion is good and perhaps we will wind up with something more complex, but better?

Automating the reporting side would make it easier to spot any abuses and to control and maintain the listing. We could have some kind of signoff on it so that the system generates the scores and a nominated person or persons has to accept the score after being presented with a report on it. Should be faster than doing manual calcs.

If the administrators can figure out ways to implement the "panel" system in an efficient and less time-consuming way (at their end), then it would be really nice.

My thought was that with a panel, this appraisal thing COULD finally get SOME recognition. (I forgot where I posted it, but I recently replied to a thread with just one line: there ain't no such thing as a professional domain appraiser.)

On the one hand, we might want to avoid falling into old AfterNIC's totally laymen type of opinion polling by setting up some parameters round the appraisers. For instance, like what I've already mentioned: membership of 3 months old with 100 posts or more; and perhaps a published e-mail or website so users can tell that these are REAL people instead of a "star chamber" of some sort.

On the other hand, we might also want to make this panel as open as possible by allowing new panelists to join once they meet our prescribed conditions (which are therefore wise to be set as necessarily high).

Of course, this may sound a bit off-track to DomeBase's original PEER review. Hmm ... what do you guys think?

P.S. I'm a bit reserved about outcasting the outlier. As DomeBase said, the medium thing has already taken care of the bias. Banning outliers might discourage genuine/honest appriasals. After all, all appraisals contain some subjective elements.

This is interesting. If we could develop a respected and useful appraisal system that feeds into a domain name showcase and is empirically tested by comparison with actual sale prices, then this would be a neat added feature for this forum.

I originally was thinking of something that would not require any added work by the mods, but if Safe is willing and not too much work, then I have a wild idea... perhaps we could design a system with two evaluations: one by a defined panel (with access based on some of the criteria NameSlave suggests); and one open evaluation by any/all forum members. This could give us the best of both worlds. Panel has advantage of less possibility of collusive appraisal inflation and consistent set of evaluators; open appraisal has advantage of getting more input and having a friendly "everyone is welcome" flavor to it. People can compare the two evaluations and give more weight to whichever one they prefer .

How about anyone can vote in the open appraisal and the following criteria for voting in the defined panel --

(1) membership in DomainState for at least three months;

(2) at least 20 posts (low, but do not want to encourage clutter); and

(3) (a) external means of identification through website or e-mail or (b) for people who really want to be anonymous but have won the trust of people on the forum, members who ask to be a panel member without identification and get a
majority in a forum vote.

This may seem a little complex now, but I think we could get into it, it could work, and hopefully could have very good results. Having both panel and open votes could be the best of both worlds?

Originally posted by nameslave

P.S. I'm a bit reserved about outcasting the outlier. As DomeBase said, the medium thing has already taken care of the bias. Banning outliers might discourage genuine/honest appriasals. After all, all appraisals contain some subjective elements.

It only takes one $50,000 point (read: jerk) to completely overcome nine $1000 points.

This is why such points are called "outliers".

Originally posted by DomeBase
perhaps we could design a system with two evaluations: one by a defined panel (with access based on some of the criteria NameSlave suggests); and one open evaluation by any/all forum members. This could give us the best of both worlds

I would strongly encourge having the panel valuate blind first, then when their results are published open up a public valuation.

In fact we now have 3 lists,

1) Current panel "speculation"
2) Current public "speculation"
3) Historical *REAL SALES* prices

Comparing 3 to 1 and 2 could be very informative!

Originally posted by ILikeInfo
It only takes one $50,000 point (read: jerk) to completely overcome nine $1000 points.

This is why such points are called "outliers".

Sorry, my typo: it's MEDIAN. (Where's my SPSS manual? I don't really need that for the median, do I! LOL!) And that's why DomeBase suggests using median instead of mean (or average).

Doh ... why does it have to have such a long name?

Originally posted by nameslave
Sorry, my typo: it's MEDIAN. (Where's my SPSS manual? I don't really need that for the median, do I! LOL!) And that's why DomeBase suggests using median instead of mean (or average).

Sorry I missed that. I still think they should get tossed though.

yes, the median should help negate impact of extremes:

vote 3
vote 4
vote 5
vote 5
vote 6
vote 6
vote 2000
------------------
ave: 290
median: 5

vote 0
vote 800
vote 805
vote 820
vote 830
vote 850
vote 870
------------------
ave: 711
median: 820