[AISWorld] A philosophical (and controversial) question on DoS on social settings ...

[mmora at securenym.net]

Dear colleagues in AISWorld:

A philosophical (and controversial) question on DoS on social settings,
and and explicit critique to top IS journals which considers DoE as the
"magic" technique for top research (e.g. MIS Q, IS Research, JMIS, etc).


Colleagues, my doctoral degree is in Systems Engineering, so I use
statistical tools like analysis tools. Well, I have a core philosophical
inquiry about the most essential adequacy of applicability of design of
experiments on social contexts. I will summarize my arguments as follows:

#1 Design of experiments (DoE) emerged in the agriculture discipline
thanks the work of Sir Ronald A. Fisher at UK. According to their library
of papers and letters -available on line at the University of Adelaide,
Australia (http://www.adelaide.edu.au/library/special/digital/fisher/),
where he worked in this last life stage- all of his research was developed
in natural or physical settings or systems. He was asked to suggest the
utilization of design of experiments by social scientists from top USA
universities, and he suggested that it could be done. However, for a
particular case on a medical application, Sir Fisher critiqued a bad
utilization of it.

#2 DoE was also further mainly used in industrial settings (Prof. George
P. Box, was one of the leaders on it, and his son-in-law).

#3 In social settings, from the 50's (mainly in Psychology area), it was
introduced such techniques for supporting a quantitative-based research
approach, and it was transferred to other areas as IS (Information
Systems), where statistical-based techniques are considered as the most
appreciated by top IS research journals.

#4 A core condition to use DoE, is the utilization of a homogenous control
and treatment groups (despite pursuing a robustness design is also a
trade-off). In physical or natural systems, this condition can be assured
easier than in social systems. When researchers select a physical
material, sections of a terrain, or even living subjects (but not human
beings) for conducting experiments, rarely they must prove such a
homogeneity in the studied groups. It is taken by grant under the
assumption of a piece of material or a set of living subjects (collected
from the same natural setting) are homogenous. However, for social systems
(where human beings are the units of study), demographic data that support
this homogeneity is required.

#5 Well, the core inquiry is the following: demographic data collected on
human beings could be insufficient to establish underlying differences and
not measured attributes, and this mistake could be similar to consider
homogenous pieces of materials only by color, area size and piece’s weight
by example, despite these can be totally different materials with
different chemical, electrical and other physical properties (an extreme
case). Under this case, the differences obtained between control and
treatment groups can be influenced by the difference of underlying and not
identified attributes rather the applied treatments.

#6 The DoE on repeated measures (to apply the series of treatments on the
same individuals) is usually suggested to avoid previous mistakes, but it
keeps still the particular attributes of human beings where contextual and
temporal-based influences can also interfere.

#7 DoE published in tops journals are excellent lab exercises but
unfortunately their results are scarcely transferred to industrial
settings as it is the usual in hard sciences. In particular, the MIS Q
papers from the 70's and 80's, while use basic statistical tools, their
analysis was mainly conducted on real settings. Thus, IT research
knowledge could easily transferred to IT practitioners. Even MIS Executive
is hard to IT practitioners.

In summary, while that DoE on physical objects can assure almost a total
replication of the results under the same physical environment (due to the
existence of natural laws, so humanity has developed highly predictive
technology), the results obtained in social systems (mainly from lab-based
experiments) rarely are replicated, producing a fragmented science. Of
course, that DoE is a powerful analysis tool for scientific research and
discovery, but my core inquiry is on the philosophical applicability on
social systems, despite of more of half century of utilization in research
social settings.

Comments and debate are welcome !

Dr. Manuel Mora
EiC of IJITSA / ACM Senior Member
Full-time Professor and Researcher
Autonomous University of Aguascalientes
Mexico

PS In 1988, I took a 8-hour course with Prof. George P. Box, invited by
Monterrey Tech, Mexico, during my MSc studies in Monterrey Tech. The
course was, actually for the industry !