[I-5] Considering a Qualitative Researcher in Esteem?

A colleague remarked to me recently that she heard it takes forever to get respected as a qualitative researcher. “Is this just an unfounded rumor?” she inquired. That is a difficult question, and I am probably one of the least prepared to answer that. So, I’ll try.

Of course, David Silverman in Interpreting Qualitative Data (2006) acknowledges the sentiment that some call qualitative researchers “journalists or ‘soft scientist’ who’s work is ‘termed unscientific, or only exploratory, or entirely personal and full of bias” Denzin and Lincoln 1994:4). This seems to support her premise in general.  Also, in the overview, the Research Methods Knowledge Base, a comprehensive web-based textbook that addresses social research methods, links lack of respect of qualitative researchers to perceived validity of research. They offer an alternate criteria for judging qualitative research (it is highly debatable) of which some of their proposals entail basing credibility from the perspective of the participant instead of externally and judging transferability of research not by remarks from the author, but rather, positioning it in the domain of the reader who has been supplied all the relevant data. However, when she asked about “being respected” as a qualitative researcher, my first response was “respected by whom”?

Her life and experiences are quite different than mine. Research was not a part of my education. I am a professor and an architect, and thus, I am in a field where, in my opinion, you gain expertise from the ‘doing’ – the art of design and the business of building. Granted, I conduct research frequently in my practice, but it is highly quantitative with comparison of proficiencies for building systems and materials. Indeed, within our 5 year bachelorate program, my students learn little about research (this is similar with Masters Programs). Also, keep in mind that a master’s degree in architecture or design is the terminal degree for professorship at nearly all universities in America.  It is interesting that at the post-graduate level in architecture and design, one often has their first formal introduction into research methods! One of my colleagues and none of my department administration have doctoral degrees. Nevertheless, although not in an educational setting, there is common exposure to qualitative methods.

If I am concerned about respect as a qualitative researcher from my peers (architects who might use the findings to aid in the creation of their work or fellow faculty or department professors),it is important to know that qualitative research methods are commonplace in the professional architecture and design literature. From the arguably foundational “House as a Mirror of Self: Exploring the Deeper Meaning of Home “ by Clare Cooper Marcus (1995), which uses an open interview method to the many published case studies by the Harvard University, such work abounds. I believe that architects and designers, in general, respect a qualitative methodology as a way to more fully explore the experiential perception of a space, but they look at it with the scrutiny of one devoid of formal research education, wary of being told “convincing stories” as David Silverman remarks in his Preface on page xiii.  

Respect from the architectural research community (AIA Committee on Architecture for Education, Interior Design Education Council, etc…) I’m sure, is more hard-earned. Now, with my entry into the PhD program, I move from being largely a recipient to a contributor of works. I hope that it will not be a matter of me being respected as a qualitative researcher, but rather, my qualitative research earning respect. 

[I-5] Perceived Research Validity and the Size of Study/Amount of Data

I often hear rumors of a perceived dichotomy between the depth of understanding and quantity of fixed data. Comments like “The number of participants involved in a study seems to indicate validity” and “this attitude might be detrimental to the initiative to improve the classroom in the minds of a single teacher”, abound.

I believe that these assumptions might be widespread. The ideal is that, as Silverman says in Interpreting Qualitative Data(2006) , “qualitative research should go for “Authenticity’ rather than sample size”. But, later he bemoaned that, “…in qualitative research, observation is not generally seen as a very important method of data collection. This is because it is difficult to conduct observational studies in large samples.” This leaves one with the impression that there is a premium in the presentation of lots of data or in conducting large group studies.

Jakob Nielsen, Ph.D., a user advocate and principal of the Nielsen Norman Group writes about “numbers fetishism” in his biweekly newsletter and admonishes against it. He advises, like David Silverman, that despite the ethos of statistics, quantity of numbers or even the use of quantitative research methods, does not automatically make research more valid. Mr. Nielson points out that if one were to study the 2004 U.S. presidential election in Florida to determine why George Bush won, working in a strictly quantitative method (with 100 persons sampled) they would not have had statistically significant evidence to indicate that some persons voted for Pat Buchanan mistakenly on the butterfly ballot (less than 1 % of the electorate did that). However, as the author states “A qualitative study, on the other hand, would likely have revealed some voters saying something like, "Okay, I want to vote for Gore, so I'm punching the second hole ... oh, wait, it looks like Buchanan's arrow points to that hole. I have to go down one for Gore's hole."

The whole idea of the size of statistical groups and relevancy reminds me of a book I read four years ago called ‘The Wisdom of Crowds’ by James Surowiecki (2004) [See audio excerpts 1,2,3,4]. Mr. Surowiecki presents research that size does matter in that large groups of people are collectively more intelligent and savvy than experts are individually. He says that there are three kinds of inquiries where this phenomenon is apparent – cognitive, cooperative and those instances requiring coordination between people. The foundation of his thesis is based on an observation by Francis Galton in 1906 in which 787 local farmers and townspersons at the annual West of England Fat Stock and Poultry Exhibition bet on the ‘slaughtered and dressed’ weight of an ox. The mean wager of the crowd was 1197 pounds. The actual dressed weight was 1198 pounds. In another instance, in May of 1968 the USS Scorpion was lost at sea in a possible area of 20 miles in diameter by thousands of feet deep. Sontag and Drew explain in their book “Blind Man’s Bluff” (2000) that against popular convention, John Craven, the naval officer in charge of locating the wreckage did not hire an expert in submarines or oceanography, but rather, he created several different scenarios of what might have happened to the ship, probable submarine speed and angle of descent. He then surveyed a large group individually as to their estimate of the plausibility of each scenario and then used Bayes’s theorem (which allows for new information and its effect on pre-existing expectations) to calculate the results. Although no one person in the group had selected the discovery point as their likely spot, the calculation of Craven’s group was only 220 yards from where submarine was recovered. Therefore, I hope Surowiecki’s conclusion of ‘individual power’ heartens the singular classroom teacher !

Hopefully, this assumption of the inherent validity of a large study population or voluminous statistics can be debunked and take with it the ethos of the intellectualist (so evidenced in the way my parent's eyes glaze over when I say the word “pedagogical”). In its place, I hope, will come an affinity for the appropriate design of the methodology to the research.

[I-5] Using Qualitative Methodology in Architectural Research?

My previous experiences have left me with the assumption that qualitative methods are suspect in the assessment of the effectiveness of designed spaces. Indeed, I believe that individual perception of the built environment is highly subjective and can be valid for each person, but design, by its very nature is ordered in an intentional way. Thus, while I believe that architecture as an art form always expresses something – the newly-built always comments on the existing environment in terms of what indigenous elements it chooses or refuses to integrate, how important it perceives itself to be, how much it chooses to aid the patron through its structure and the hierarchy it gives to particular spaces, how responsive it chooses to be to cultural concerns, and in many others ways (and considering how easy it is to obscure, in these anthropomorphic terms, the direct intention of the school board, principal or regent),the pressure and breadth of these influences are largely in the mind of the beholder.

Therefore, in analyzing my thoughts about what it means to use qualitative methods, I find that I have approached articles with a general skepticism of validity when they (and this is so typical in architectural journals) cite anecdotal support for particular design elements employed in a prototypical school, loosely based on a construct of a Model. Or, in regard to qualitative research methodologies, I cringe when I read that specific educational facilities are evaluated to parse out just what are the influences at work in synthesizing an innovative pedagogical and organizational school design into a building concept and find that they used only open ended surveys, participant observation and relative research review to support their findings. These often conclude with an attempt to apply it to general situations. And case studies abound! One can find studies that certain colors enhance learning, spatial geometry is directly causal in scholastic proficiency and one designer remarked that the theory of Constructivism is evidenced in his facility in that the building “provides a forum for architecture itself to become a teacher for students, as students interact... with the physical spaces we call schools”.

It is far more comfortable for me to lean toward “findings based on ‘reliable’ variables”, as David Silverman puts it, especially in the face of a facility that will cost several million dollars to construct and furnish and take many months to build.
Our readings do enlighten my initial conjecture, however. The author’s discussion of using qualitative data as a pre-test certainly seems apropos. His declaration that “Whatever we observe is impregnated by assumptions” and the validity of a functioning hypothesis was insightful to me. Furthermore, the concern to integrate learning and environment is so complex that more than one method may be required and quantitative methods alone may not provide meaningful answers to the design and assessment of learning environment in an experiential way. So now, I am less apt to think that one is ‘good’ and one is ‘bad’.

Perhaps when all is considered, credibility, validity and other challenges in addressing whether to use qualitative methods, quantitative methods or a combination, seem daunting, because they are, and such a journey for me is best undertaken open and with prior assumptions questioned.