Evaluating Literature and Methodological Strategies for Business Research

Part A- Review of Four Articles

Golicic, S. & Mentzer, J. (2005) Exploring the drivers of interorganizational relationship magnitude, Journal of Business Logistics, Vol 26(2) pp.47-71

The study authored by Golicic & Mentzer, 2005, aims to investigate what factors contribute to various levels of relationship magnitude and attempts to explore contextual drivers influencing relationship magnitude. Golicic, Foggin and Mentzer cited in Golicic & Mentzer (2005) define relationship magnitude as; “the degree or extent of closeness or strength of the relationship amongst organisations” (p. 47).

Areas in the literature that have been critiqued include historical views on the structure of relationships broken down into two main components; the economic structure (‘type’) and policy in terms of strength and closeness of relationships (‘magnitude’). Additionally the authors critique the business press who have advocated the need for firms to build and maintain closer, long term relationships with suppliers and customers. A number of different studies are cited supporting the notion that some firms don’t want or need these closer relationships.

The methodology used to collect data for the research was a qualitative method of content analysis of depth interviews.

The results from the content analysis of depth interviews identified eight contextual drivers which the authors note as; capabilities, expected benefits, external influence, history, importance, interpersonal interaction, performance and strategy congruence. The authors discuss the drivers in detail and underpin their findings with further literature support but make the distinction that only in specific cases do the drivers relate to magnitude of a relationship over type.

An opportunity for further research identified by the authors is in developing further hypotheses for the contextual drivers that have been presented and testing these for validity in a manner that is perhaps more robust than the qualitative analysis presented.

Jackson, S. Wilson, J. McCarthy, B. (2004) A new model of scheduling in manufacturing: tasks, roles and monitoring, Human Factors, Vol 46(3)

The paper authored by Jackson et al. 2004, aims to present a new production scheduling model representative of the significant facilitation and implementation aspects of human scheduling that is ignored by many computational generated scheduling models.

Areas in the literature that have been critiqued include research that has previously attempted to develop models of human scheduling in manufacturing contexts and research that has explored the scheduler activities of task, roles and monitoring.

The authors present a descriptive model of scheduling performance that captures scheduler behaviour and performance considered more appropriate than previous theoretical scheduling models.

The methodology used for collecting data to present the new scheduling model was a qualitative field study comprising of two techniques; direct observation and retrospective decision probes.

The results indicated that schedulers presented three distinct types of activities in their jobs, namely; tasks, roles, and monitoring. Another key finding was that schedulers do not carry out all tasks in a sequential manner all of the time and that scheduling roles change over time because of external factors that influence the scheduling function, such as uncertainty and inherent instability of the business environment.

An area identified for further research was to establish how the new model of scheduling may be utilised in the context of both existing and new production planning & control and enterprise information systems.

Ritschel, D. Greiner, M. Reynolds, D. Seibel, M. (2003) Flyaway costs versus individual components of aircraft: An analysis, Air Force Journal of Logistics, Vol 27(4) 30 pp.32-47

The article authored by Ritschel et al. aims to investigate and measure the risks associated with taking a macro versus micro approach to aircraft cost estimation. The fidelity of a cost estimate developed at the flyaway cost level (macro) versus at the individual component level (micro) is analysed in order to provide program manager’s with guidelines for appropriate allocation of cost analysts in today’s constrained environment.

The research attempts to address the following questions:

1. Which aircraft components have the most cost-estimation error risk and what is that risk?
2. What is the cost estimation error risk associated with estimating at the flyaway cost level?
3. Is there a statistically significant difference in estimating at the component level versus flyaway level?
4. Given a constrained resource environment, where should cost analysts focus their attention when developing an aircraft cost estimate?

The research critiques previous literature that focuses on fundamental components and techniques used to develop an aircraft cost estimate for two categories of aircraft; fighters and intertheater aircraft (used for supply and transportation). The components and techniques reviewed are the Work Breakdown Structure (WBS), Cost Estimating Relationships and Aircraft Estimation Techniques encompassing parametric estimation and grassroots estimation.

The methodology used for collecting data was an economic model.

A key point identified by the authors with regards to the results is the need for a practical application perspective to be considered; although there is a statistically significant difference, the estimation errors from the models are extremely small in real terms when considering the multi-million dollar costs of aircraft weapons systems. In concluding, the authors contend that the error risk is not large enough for program managers to be concerned when allocating resources and they will continue to allocate based on other considerations such as time constraints.

A number of areas related to the methodology are identified by the authors for future research. A couple of these can be summarised as follows; examine the non-recurring estimating error between differing WBS levels and; consider applying the methodology to different weapon systems other than aircraft.

Subramaniam, C. (2004) Human Factors influencing fire safety measures, Disaster Prevention and Management, Vol 13(2) pp.110-116

The study authored by Subramaniam (2004), aims to identify the current fire safety condition in residential colleges located in a Malaysian university. It also seeks to identify important human involvement elements that need highlighting to achieve a higher standard of fire safety management in residential colleges.

The research does not provide an in-depth critical review of previous literature. This may be due to the quantitative methodology whereby the author has assumed that site survey’s will satisfy validity requirements by collecting factual evidence in the field. An inference to a previous study on fire safety management and risk assessment is provided in the introduction (Ramachandran, 1999), however the author does not acknowledge that his research is an extension of this study. Other citing’s noted pertain to papers discussing human behaviour in emergency situations but again these are not critically reviewed.

The quantitative methodologies used to collect data took the form of a site survey and a questionnaire survey.

The results indicated that the overall current fire safety condition within the 5 residential colleges surveyed was 76 percent compliant and that the most critical non-compliant item was exit sign requirements to meet current Malaysian Building Bylaws. Additionally, the analysis revealed there were positive relationships between pre-disposing factors and fire safety behaviour and lifestyle, whereby predisposing factors concern a person’s beliefs, values and motivation towards change.

In concluding, the author contends that the study provided will enhance further understanding of human involvement in fire safety, however there are no suggestions for future research made.

Part B- Analysis of the Strengths and Weaknesses of Research Methodologies Used

Qualitative Field Study- Jackson, Wilson and McCarthy (2004)

A qualitative field study approach consisting of two methods was utilised; direct observation and retrospective decision probes. These were undertaken on 7 schedulers within 4 manufacturing environments.

Direct observation is a method of collecting evaluative information in which the evaluator watches the subject in his or her usual environment without altering that environment (Holmes, 2013). The authors utilised this method to observe what schedulers do and how they do it in their usual scheduling environment.

Retrospective decision probing is a probing technique employed to gain subject feedback on why certain decisions are made in hindsight (Crawford et al., 1999). After completing daily observations, the authors used this method to ask schedulers a series of interview questions about the decisions they’d made in the course of their duties. The same series of questions were asked of each scheduler for reliability and to maintain a structured decision probe format and for reliability. The responses were populated into a decision diagram for further analysis.

Direct Observation

Strengths

• Useful for framing hypotheses. By observing a phenomenon continuously, the researcher is able to get well acquainted with the subjects observed.
• Access to subjects in a natural ‘real life’ setting provides an accurate representation of the subjects under research
• Facilitates an ability to take very detailed notes
• Able to explain behaviour and communication patterns in ways that a survey, interview, or experimental design cannot.
• Compatibility with many other research methods and can be a strong component in more complex research designs.
• Requires less effort on the part of the subjects being observed.

Weaknesses

• Can be viewed as too subjective due to researcher interpretations.
• Time consuming.
• Requires varying levels of effort on the researcher’s behalf to arrange suitable access to subjects.
• If overt observation, may affect subject behaviour and validity of findings.
• If covert observation, may attract ethical issues.
• Sheer volume and detail of notes can be cumbersome to organise.
• Findings may be restrictive due to a relatively small sample size under observation.

(Lashley, 2017)

Retrospective Decision Probing

Strengths

• Can provide clarity to why subjects made the decisions that they did.

• Probing questions allow the researcher to obtain targeted information.
• Doesn’t interfere with subjects performing in their natural environment because decisions are probed in hindsight.

Weaknesses

• Time consuming.
• Subjects may have difficulty remembering why they made a particular decision at the time the decision was noted due to being probed in hindsight
• Potential for bias. The use of probes may lead the subject to a particular type of response.

(Willis,1999)

Economic Modelling- Ritschel et al. (2003)

The quantitative methodology used for collecting data was an economic model. An economic model can be defined as a “hypothetical construct that embodies economic procedures using a set of variables in logical and/or quantitative correlations.” (Francis 2018).

The author’s commenced analysis by segregating the aircraft cost data into the aircraft category subsets of ‘fighter’ and ‘intertheater airlift’ and by their macro and micro components of ‘flyaway cost’, ‘basic airframe’, and ‘other air vehicle’. Next, a total of six multiple regression equations were developed for each of these categories. A Monte Carlo simulation was then applied to these regression equations. Specifically, the bootstrap technique was used to estimate the standard error of the equations. The resulting distribution from the differences of the standard error of the micro (basic airframe and other air vehicle) versus macro (flyaway cost) equations were analysed to answer the original research questions.

Strengths

• Provides a logical abstract template to help organise the analyst’s thoughts.
• The model helps the analyst logically isolate and sort out complicated chains of cause and effect and influence between the numerous interacting elements.
• Facilitates logical experiment, producing different scenarios, attempting to evaluate the effect of alternative options, or weighing the logical integrity of arguments presented in prose.
• Can have visual appeal when presenting an argument.

Weaknesses

• If initial assumptions are wrong or misleading, or even incomplete, despite the logical integrity of the model, the model’s conclusions will be as much in error as the initial assumptions.
• Can lead to oversimplification of real world views based on the variables omitted from the model that do actually matter in reality
• Models that are mathematical may also be limited because they must be tractable. In other words, they are useless unless they can be solved or manipulated to produce insightful results

(Evans, 1997)

Content Analysis of Depth Interviews- Golicic and Mentzer (2005)

A qualitative methodology was employed to collect data in the form of content analysis of depth interviews.14 depth interviews were conducted with employees from four interconnected supply chain firms in three different industries (automotive, pharmaceutical and plastics).

This methodology was selected due to limited previous research on relationship magnitude and is considered useful for studying beliefs, organisations, attitudes and human communications associated with relationship magnitude. The authors adapted an eight step method;

1) identify the questions to be asked and constructs to be used;

2) choose the texts to be examined;

3) specify the unit of analysis;

4) determine the categories, or themes of meaning, into which responses are divided;

5) generate a coding scheme or coding rules;

6) conduct a sample or pilot study;

7) collect the data and revise the scheme as necessary; and

8) analyse the data and assess validity and reliability. 

(Harris, 2001)

Depth interviews or in-depth interviews can be defined as a form of non-standard or semi-structured oral interviews that afford relatively large freedom on the interviewer’s behalf with regard to content and design (Harris 2001).

Content analysis can be defined as the study of recorded human communications and is essentially a coding operation process that transforms raw data into a standardised format (Weber 1990).

Content Analysis

Strengths

• Looks directly at communication via texts or transcripts and targets a central aspect of social interaction.
• Allows a closeness to text which can alternate between specific categories and relationships and statistically analyses the coded form of the text.
• It is an unobtrusive means of analysing interactions.

(Writing @ CSU | The Writing Studio, 2018)

Weaknesses

• Time consuming
• Subject to increased error, particularly when relational analysis is used to attain a higher level of interpretation
• Can be devoid of theoretical base by attempting to loosely draw meaningful inferences about the relationships and impacts implied in a study
• Often disregards the context that produced the text.

(Writing @ CSU | The Writing Studio, 2018)

Depth Interviews

Strengths

• Interviewers can establish rapport with participants to make them feel more comfortable and at-ease, which can generate more insightful responses, especially regarding sensitive topics.
• Interviewers have much more opportunity to ask follow-up questions, probe for additional information and circle back to key questions later on in the interview to generate a rich understanding of attitudes, beliefs, perceptions, motivations, etc.
• Researchers need fewer participants to obtain useful and relevant insights.

Weaknesses

• Time consuming, as interviews must be transcribed, organised, analysed and reported.
• Sheer volume and detail of notes can be cumbersome to organise.
• Participants must be carefully chosen to avoid bias, which can result in a longer vetting process.

(Steber, 2017)

Surveys- Subramaniam (2004)

The methodology used to collect data for research was in two formats; site survey undertaken on a sample of 5 residential colleges within one university; and a questionnaire survey issued to a sample population.

An audit checklist was adapted from Ibrahim (2000) for the site survey that focused on five main safety features installed in residential colleges:

(1) Portable fire extinguishers

(2) Hose reel systems

(3) Fire alarm system

(4) Exit sign

(5) Staircase

The questionnaire survey was utilised to assist with gathering further quantitative data that was divided into five sections for soliciting information:

S1. Fire safety lifestyle and behaviour

S2. Predisposing factors

S3. Reinforcing factors

S4. Enabling factors

S5. Demographic data.

Questionnaire surveys are widely accepted to be at the heart of surveying as a quantitative research method. The strengths and weaknesses of a questionnaire survey are noted here:

Strengths

• Ability to generalise about an entire population by drawing inferences based on
• When implemented properly, the sample survey is a reasonably accurate method of collecting data.
• Offers an opportunity to reveal the characteristics of institutions and communities that represent these entities in a relatively unbiased and scientifically rigorous manner.
• Can be implemented in a timely fashion so that the actual data gathering is performed in a relatively short period of time.
• Generation of standardised data that is extremely amenable to quantification, computerisation and statistical analysis.
• Replicability in that a questionnaire used in one city or community can be implemented in another community or administered once again in the same community.

(Rea and Parker, 2014)

Weaknesses

• Can be expensive
• Time intensive
• Survey design requires training to avoid bias
• Difficult to develop rapport
• Validity depends on the response rate.

(Rea and Parker, 2014)

PART C- Research Topic and Methods Used to Explore Topic

The research topic I have selected is ‘Delivering Successful Public Sector Tertiary Health Projects: A Comparative Analysis of the Managing Contractor Model (MC) Versus the Public Private Partnership Model. (PPP).

My research sets out to address one primary question / objective within an Australian context:

Is one project delivery model more successful than the other in terms of delivering on the traditional project parameters of time, cost and quality and if so what evidence is there to support this?

The methods I will use to implement my research and why I have selected these follow:

• A critical literature review of the two project delivery methods will be undertaken. This may require critiquing research on MC and PPP projects in other countries to put my research into a wider context with others and to determine what has already been studied in this area.
• I intend to undertake Qualitative research in the form of content analysis of case studies. Stern (1980) notes qualitative methods are ideally suited to research substantive areas about which little is known. Case study analyses are one method of qualitative research that is especially effective in approaching phenomena that might be ambiguous and part of a dynamic process (Thorpe & Holt, 2011).
• The case study analyses could potentially include any relevant State Government Auditor General’s Reports (AGR’s) that concern public sector tertiary health projects delivered using MC or PPP methodology in the last 20 years. Auditor general reports are an excellent source of information related to the performance of how effectively public sector agencies provide services and use public money. The reports make recommendations that promote accountability and transparency in government and improvements in service efficiency and effectiveness (VAGO, 2018). By reviewing any relevant AGR’s, I hope to be able to obtain further data that validates the success (or failure) of MC and PPP tertiary health projects undertaken by State Government’s.
• Given the parameters of time, cost and quality (i.e. conformance with specifications) can be attributed to project success, I will give consideration to employing some quantitative analyses in my research that may be in the form of statistical assessment.

References

• Crawford, S., MacCarthy, B. L., Wilson, J., Vernon C. (1999), Cognition, Technology & Work, viewed 1 October 2018, https://link-springer-com.ezproxy.lib.swin.edu.au/content/pdf/10.1007%2Fs101110050033.pdf
• Evans G. (1997), Economic Models, viewed 2 October 2018, http://pages.hmc.edu/evans/chap1.pdf
• Francis J. (2018), What is an Economic Model? – Definition & Example, viewed 2 October 2018, https://study.com/academy/lesson/what-is-an-economic-model-definition-example.html 
• Golicic, S. and Mentzer J., (2005), Exploring the drivers of interorganizational relationship magnitude, Journal of Business Logistics, Vol 26(2) pp.47-71
• Harris, H. (2001), Content analysis of secondary data: A study of courage in managerial decision making, viewed 3 October 2018, https://search-proquest-com.ezproxy.lib.swin.edu.au/docview/38348777?rfr_id=info%3Axri%2Fsid%3Aprimo
• Holmes A. (2013), ‘Direct Observation’, in Volkmar F. R. (eds), Encyclopedia of Autism Spectrum Disorders, Springer, New York, NY, pp. 980
• Ibrahim, H. (2000), ‘Assessment of active fire protection for government boarding school’, Unpublished masters dissertation, University Putra, Malaysia
• Lashley, M. C. (2017), ‘Observational Research, Advantages and Disadvantages’, in Allen M. (ed), The Sage Encyclopaedia of Communication Research Methods, Sage Publications Inc, Thousand Oaks, CA, pp. 2-5
• Ramachandran, G. (1999), Fire safety management and risk assessment, Facilities, Vol. 17, pp. 363-76.
• Rea, L. M. and Parker R. A., (2014), Designing and Conducting Survey Research: A Comprehensive Guide, 4^th Ed. viewed 4^th October 2018, https://proquest-safaribooksonline-com.ezproxy.lib.swin.edu.au/book/social-sciences/9781118767023/part-one-developing-and-administering-questionnaires/chapter02_xhtml
• Ritschel D., Greiner M., Reynolds D., Seibel M., (2003), Flyaway costs versus individual components of aircraft: An analysis, Air Force Journal of Logistics, Vol 27(4) 30 pp.32-47
• Steber, C. (2017), In-Depth Interviews: Data Collection Advantages and Disadvantages, viewed 3 October 2018, https://www.cfrinc.net/cfrblog/in-depth-interviews-data-collection-advantages-and-disadvantages
• Stern, P. N. (1980), Grounded Theory Methodology: Its Uses and Processes, Image, Vol 12
• Subramaniam, C., (2004), Human Factors influencing fire safety measures, Disaster Prevention and Management, Vol. 13(2) pp.110-116
• Thorpe, R. and Holt, R., (2011), The Sage dictionary of qualitative management research, viewed 10 October 2018,http://methods.sagepub.com.ezproxy.lib.swin.edu.au/Reference/the-sage-dictionary-of-qualitative-management-research/n14.xml
• Victorian Auditor General Office, (2018), viewed 10 October 2018, https://www.audit.vic.gov.au/our-role
• Weber, R. P. (1990), Basic Content Analysis, 2^nd Ed. Sage Publications, Newbury Park, CA
• Willis G. B. (1999), Cognitive Interviewing A “How To” Guide, viewed 1 October 2018, http://www.chime.ucla.edu/publications/docs/cognitive%20interviewing%20guide.pdf
• Writing @ CSU | The Writing Studio (2018), Advantages of Content Analysis, viewed 3 October, https://writing.colostate.edu/guides/page.cfm?pageid=1318&guideid=61
• Writing @ CSU | The Writing Studio (2018), Disadvantages of Content Analysis, viewed 3 October 2018, https://writing.colostate.edu/guides/page.cfm?pageid=1319&guideid=61