CALIopolis

I am so excited to make this post.

Here's the press release.

What does the new website, Avvo, which rates lawyers and CALI Awards have to do with each other?

Read on...

A new website, Avvo (shortened "avvocato" which is italian for lawyer) purports to rank lawyers based on objective algorithmic evidence...

"...scores are calculated using a mathematical model, all lawyers are judged by the same standards. The Avvo Rating takes into account many factors, including experience, professional achievements, and disciplinary sanctions..."

I looked up some famous lawyers I know about and they did indeed have a lower score. For example, see "Fast Eddie" Vrdolyak's rating here. Mr. Vrdolyak was a longtime alderman in the Chicago City Council and was recently indicted on fraud charges. This wasn't the reason for his low score - rather he had been sanctioned by the ARDC in the past.

Of more personal interest, a commenter on Slashdot noted that they have a higher score due to the CALI Awards they received in law school. Law students receive a CALI for achieving the highest grade in a course. Looks like Avvo's been crawling the CALI website.

Only half the law schools in the U.S. participate in the CALI Excellence for the Future Awards program. Perhaps I will get a few more emails this week.

A few years back, someone started posting death rates for hospitals and doctors on the web. It was a simple statistic, but it's one of those statistics that is easy to misinterpret out of context. Some doctors just have sicker patients, right?

I was wondering why no one seems to have posted attorney ratings based on wins/losses in court cases. Wouldn't you like to know if your attorney was a "winner"?

Avvo doesn't do that...yet.

The reason for the Slashdot story is that an attorney is suing Avvo for his low rating. It looks like the data that Avvo is using is publicly available information, which would be an argument in their favor, I presume (IANAL).

More relevant, Avvo lets former clients post reviews of their experiences with attorneys. Sort of like an Angie's List for lawyers (though Angie doesn't have any listings or ratings for lawyers). Why shouldn't clients rate lawyers like they do plumbers, carpenters and dog-walkers?

It looks like someone has thought about suing Angie's List for a poor rating posted by a user, but ...

"... he'd like to sue Angie's List but that his attorney tells him it's protected..."

Angie's List doesn't rate vendors though - the users do. Angie just aggregates the information.

I can certainly see a downside to non-contextual rankings and sites like RateMyProfessor.com and the US News Rankings of law schools have both been villified for their lack of context (former) or opaque/unfair methodology (latter).

Still, more information is better and the web is certainly all about more information.

Right?

It will be interesting to see where this goes.

Techcrunch mentions a new service called Usphere where undergrads can pay $65 to apply to 33 unnamed colleges. The colleges then send acceptances letters (or not) with the costs of tuition that the student would pay.

LSAC (Law School Admissions Council and purveyors of the LSAT) has had a service for a long time where law students can fill out a single application and have it transmitted to many (most?) law schools that participate in the program. It's a single-sign-on idea applied to law school applications which are pretty similar from school to school.

What if they took it one step further and like Usphere allowed law schools to affirmatively "bid" for students. This already happens to a certain extent - schools compete for students - but the current system does not expose a law school applicants application to every law school. Applicants pick a small number of schools to apply to and the costs of applying to law school is not trivial (several hundred dollars in many cases).

My idea is a riff on Usphere where law schools can view all the applications whether the student has indicated they want to apply or not. This would allow them to approach students that might not otherwise have applied. I believe it could be handled smoothly without generating a boatload of unwanted spam. Schools could indicate a willingness to make an offer and applicants could see who is looking at them.

The network statistics from this would also be amazingly interesting. What kinds of applications result in what kinds of offers? What types of applicants are schools looking for?

As the Techcrunch article points out about USphere, this is more like a Match.com or dating service where personality traits and desired personality traits are matched up .... maybe more like an eHarmony for law school admissions.

It would be difficult to predict if law schools would participate or if there would be any benefit. It sure seems like it would be a good idea. It's a way to escapte the "tyranny" of measuring applicant quality by only LSAT score, GPA and undergrad institution rank.

Professor Leiter is conducting a study of law faculty scholarly productivity (methodology here). The goal (or one goal) is...

"...what is the most effective and efficient way to measure the scholarly impact of a law faculty..."

What if you replace the word "scholarly" with "teaching"?

If you have seen this video...

...then you can imagine how hard it is to measure teaching impact in the same way as Professor Leiter is measuring scholarly impact but perhaps there are some objective numbers that can be discovered and measured.

• number of classes taught
• number of students in those classes
• number of credit hours those classes are worth

These can be culled from the course catalog and a the class sizes by survey.

• casebooks published (first time)
• casebooks updated
• supplemental materials published

Theoretically, we could find this information out from the commercial publisher websites and matching up authors and co-authors with the publication dates to get an average per year.

Harder to get would be...

• adoption rates for casebooks
• sales figures for casebooks and supplemental materials
• whether supplemental materials are required, suggested or recommended.

Some of this could be culled from the course syllabi - many of which are online. There is a certain amount of "everyone knows" regarding what are the most popular casebooks in the major subject areas, but the long tail is harder to measure.

Having a casebook in the marketplace will generate a certain amount of traffic for the authors who must "teach the teachers" as well. This illustrates the point that teachers teach students and teachers teach teachers. Should we measure both? It would seem that teaching teachers has more downstream impact than teaching students. The places that teachers teach teachers include presentations given relating to teaching at ...

• AALS
• Conferences
• Workshops

What about non-traditional supplemental materials?

• CALI lessons written
• Websites related to educational activities
• Blogs and podcasts with educational intent like Classcaster

I can certainly measure the first item and there are over 100 law professors who have written a CALI lesson at some time in the past, but the others would be much harder to track. A lot of web-based educational material is behind a password or inside a learning management system.

None of this speaks to quality, however and so we look back to Leiter's study. He measures quality by citation. If someone else (either a court or another faculty member) has made a citation to your work, it is an indicator of impact and therefor some measure of the works value.

The nearest thing to citation in teaching is casebook/material adoption, but this would not capture the whole picture.

It would seem that potential law students would be very interested in this type of information, especially in the 150 law schools that are not going to end up in Professor Leiter's Top 30 ranking of scholarly impact.