The Freakonomics blog reports on a new study showing that there is a narrowing gap in happiness between black and whites. The reason for this may be the lessening of “day-to-day racism” – but there needs to be more research so that scholars can figure out “How best to get a handle on the evolution of day-to-day racism.”
Tag Archives: methodology
Ranking universities around the world
Ranking American colleges and universities has become a lucrative industry with a number of publications developing rankings. This quest has been extended to ranking universities around the world but there is tension between two of the important rankings and their methodologies:
Times Higher Education produced rankings for the first time this year without the collaboration of Quacquarelli Symonds Limited. Along with the Shanghai Jiao Tong University rankings, the World University Rankings that Times Higher Education and QS published together from 2004 until last year have become the most closely watched and influential university rankings in the world.
Quacquarelli Symonds has continued to produce those rankings, now called the QS World University Rankings, and is partnering with U.S. News and World Report for their publication in the United States.
The relationship between the former collaborators has deteriorated into barely veiled animosity.
While there may be some issues between those who produce the two sets of rankings, the Times Higher Education claims it has an improved methodology which emphasizes different features of a school:
Foremost among the criticisms of the previous compilation was that it relied too heavily on a reputational survey of academics, based on fewer than 4,000 responses in 2009. THE’s new methodology is based on 13 indicators in five broad performance categories—teaching (weighted 30 percent); research influence as measured in citations (32.5 percent); research, based on volume, income, and reputation (30 percent); internationalization, based on student and staff ratios (5 percent); and knowledge transfer and innovation based on industry income (2.5 percent).
Times Higher Education said that the new system was the only global ranking to devote a section to teaching. The new methodology is much more evidence-based and relies far less on subjective criteria than the old tables, said Mr. Baty. But whereas teaching was previously measured based solely on student-staff ratio, the new rankings incorporate a reputational survey.
Which rankings system is “better” will be decided by a variety of actors. Of course, the rankings will likely differ because of the different weights placed on different university traits. The newest rankings from Times Higher Education are favorable toward American schools and UC-Berkeley made quite a jump up.
Overall, these rankings can be very powerful, not just for prestige purposes but also for helping to determining the global flow of students and money. Additionally, the companies producing and publishing the rankings have a financial stake in how the rankings are perceived both by schools and the public.
On list of generous nations, US ranks 5th
Gallup has released “The World Giving Index 2010” and the United States is tied for fifth with Switzerland and behind Australia and New Zealand (tied for first) and Ireland and Canada (tied for third).
It looks like respondents were asked whether they did three things within the past month: gave money to an organization, volunteered for an organization, or helped someone they didn’t know.
Gallup suggests “the level of satisfaction or happiness of the population is emerging as the key driver for increasing the giving of money.” They also argue there could be “a positive cycle of giving” where happier people give to others who then are more likely to give.
I would be interested to know how much a country’s culture affects this. Are there certain societal traits that lead to more giving? Or are there certain economic and governmental structures that encourage more giving?
Measuring Presidential popularity with merchandise
There are traditional ways to measure Presidential popularity: polls that in some way measure approval or disapproval. Here is another possible way: sales of Presidential merchandise.
I’ve always wondered why Presidents or other political officials allow such merchandizing using their figures and words in order to make money. Perhaps it is simply publicity (even if it is in opposition to them). Or perhaps they don’t want to appear to be the politicians who cracks down on such things. Or perhaps by running for or entering public office, there is a tacit understanding that they are now in the public eye and can be used for money-making purposes.
And what does it mean culturally to reduce any politician to a piece of merchandise?
Discussions about student-learning outcomes among college boards
As discussions about assessment and student-learning outcomes build on college campuses, a new report looks at what governing boards think about their discussions of student-learning outcomes:
While oversight of educational quality is a critical responsibility of college boards of trustees, a majority of trustees and chief academic officers say boards do not spend enough time discussing student-learning outcomes, and more than a third say boards do not understand how student learning is assessed, says a report issued on Thursday by the Association of Governing Boards of Universities and Colleges.
According to members, boards tend to focus on business matters. But this issue of assessment and student-learning outcomes is one that is likely to affect all levels of colleges and universities.
(A note about how the results were obtained: the survey was sent to “1,300 chief academic officers and chairs of board committees on academic affairs how boards oversee academic quality.” The response rate was only 38%.)
Untangling the effects of income on happiness
Examining the relationship between income and happiness can be tricky. A recent research study, conducted by two Princeton researchers and summarized by LiveScience, is illustrative of some of the issues in this research field:
-The researchers were working with a large dataset that is built around a daily survey of Americans: “they analyzed more than 450,000 responses to the Gallup-Healthways Well-Being Index, a daily survey of 1,000 U.S. residents conducted by the Gallup Organization.”
-Changes in income were measured in terms of percentages rather than absolute numbers. This was done to reflect the fact that a percentage change in income would be better for comparisons across income types. As the researchers note: ““In the context of income, a $100 raise does not have the same significance for a financial services executive as for an individual earning the minimum wage, but a doubling of their respective incomes might have a similar impact on both.”
-Survey respondents answered questions related to two measures of happiness: overall life satisfaction and what their emotions were the day before. According to the LiveScience article: “For life evaluation, participants indicated on a scale from zero to 10, from worst to best possible, how they would rate their lives. For emotional well-being, participants answered yes/no questions about whether they had experienced various positive and negative emotions a lot during the prior day.” Having both of these dimensions is critical as a general question about happiness might be interpreted differently (do the reseachers mean happy right now or overall?) by respondents.
-Some of the findings: having a “Low income seemed to magnify the emotional pain of life’s misfortunes, including divorce, illness and loneliness.” However, there was a tipping point of $75,000 where having more money didn’t help improve one’s well-being:
The researchers suggest that making anything more than $75,000 no longer improves a person’s ability to spend time with friends, avoid pain and disease and enjoy leisure time – all factors involved in emotional well-being.
“It also is likely that when income rises beyond this value, the increased ability to purchase positive experiences is balanced, on average, by some negative effects,” they write. For instance, a past study revealed a link between high income and a reduced ability to savor small pleasures, the researchers noted.
This tipping point of $75,000 is above the median income in the United States. I would be curious to know if individuals feel this tipping point when their income does rise to this level – are they cognizant of this point? Or once they reach $75,000, are they still locked into a mindset that having more money will lead to increasing levels of well-being?
Also, this $75,000 point could be quite fluid. Over time, this point would change based on economic conditions and cultural understandings of what is a “good income.”
The poor cleaniness of home kitchens
Occasionally, one can find stories about how dirty homes can be. Here is more evidence, this time regarding unclean kitchens:
The small study from California’s Los Angeles County found that only 61 percent of home kitchens would get an A or B if put through the rigors of a restaurant inspection. At least 14 percent would fail — not even getting a C.
In comparison, nearly all Los Angeles County restaurants — 98 percent — get A or B scores each year.
On its own, these are interesting results: restaurant kitchens are generally more clean than home kitchens. But there is more to this story: how exactly researchers found out about the kitchens.
The study, released Thursday, is believed to be one of the first to offer a sizable assessment of food safety in private homes. But the researchers admit the way it was done is hardly perfect.
The results are based not on actual inspections, but on an Internet quiz taken by about 13,000 adults .
So it’s hard to use it to compare the conditions in home kitchens to those in restaurants, which involve trained inspectors giving objective assessments of dirt, pests, and food storage and handling practices.
What’s more, experts don’t believe the study is representative of all households, because people who are more interested and conscientious about food safety are more likely to take the quiz.
A more comprehensive look would probably find that an even smaller percentage of home kitchens would do well in a restaurant inspection, he suggested.
On one hand, this sounds like innovative research that is the first to provide a broad overview of the cleanliness of American kitchens. On the other hand, the way the data was collected suggests one should be wary about making definitive conclusions.
The online quiz is also reliant on self-reporting.
LA Times portal on value-added analysis of teachers
The Los Angeles Times has put together an information and opinion filled portal regarding their recent publication of a value-added analysis of Los Angeles teachers.
Measuring teacher performance is a tricky subject as there are a number of factors at play in a student’s academic performance. In an article, the newspaper summarizes how value-added scores are estimated:
Value-added estimates the effectiveness of a teacher by looking at the test scores of his students. Each student’s past test performance is used to project his performance in the future. The difference between the child’s actual and projected results is the estimated “value” that the teacher added or subtracted during the year. The teacher’s rating reflects his average results after teaching a statistically reliable number of students.
In addition to these methodological questions, there are number of other fascinating issues: should this sort of information be publicly available and how will affect teacher’s performance? Is it an accurate assessment of what teachers do? What should be done for the teachers who fall outside the normal range? How will the politics of all of this play out?
For those interested in education and measuring outcomes, this all makes for interesting reading.
(As a side note: I can only imagine what discussions would ensure if similar information was published regarding college professors.)
Criteria in the college rating process across publications
There are numerous publications that rate colleges. According to this story and very helpful graphic in The Chronicle of Higher Education, publications tend not to use the same criteria:
That indicates a lack of agreement among them on what defines quality. Much of the emphasis is on “input measures” such as student selectivity, faculty-student ratio, and retention of freshmen. Except for graduation rates, almost no “outcome measures,” such as whether a student comes out prepared to succeed in the work force, are used.
This suggests each publication is measuring something different as their overall scores have different inputs. This is a classic measurement issue: each publication is operationalizing “college quality” in a different way.
The suggestion about using student outcomes as a criteria is a good one. How much different would the rankings look if this were taken into account? And isn’t this what administrators, faculty, and students are really concerned about? While students and families may worry about the outcome of jobs, I’m sure faculty want to know that their students are learning and maturing.
WEIRD (Western, education, industrialized, rich, democratic) people may indeed be weird
A new article in Brain and Behavioral Sciences makes a thought-provoking cross-cultural conclusion about WEIRD people:
The article, titled “The weirdest people in the world?”, appears in the current issue of the journal Brain and Behavioral Sciences. Dr. Henrich and co-authors Steven Heine and Ara Norenzayan argue that life-long members of societies that are Western, educated, industrialized, rich, democratic — people who are WEIRD — see the world in ways that are alien from the rest of the human family. The UBC trio have come to the controversial conclusion that, say, the Machiguenga are not psychological outliers among humanity. We are…
WEIRD people, the UBC researchers argue, have unusual ideas of fairness, are more individualistic and less conformist than other people. In many of these respects, Americans are the most “extreme” Westerners, especially young ones. And educated Americans are even more extremely WEIRD than uneducated ones…
One of the consequences of this argument that is pointed out by the authors is that WEIRD people are then a bad population for studies and experiments because the results may not be generalizable.
I wonder how average Westerners and Americans in particular would react after reading this argument. Perhaps it might fit in with some of the ideas regarding “American exceptionalism” – though whether this is good or bad could be debated.
Regardless, if other researchers agree with these conclusions, it suggests that social science studies about humanity need to be expanded across the globe. The era of the undergraduate research subject might then be over.
Legally Sociable 