Reverse step: building research and knowledge from data or observation study

Research is an iterative process and can be performed in either forward or reverse steps. Different fields of research require different research steps that best suit the specific situation of each field.

An important note is that both research step approaches are an iterative process.

Another important thing is that the best model or theory that fits general cases is the one that is “simple” (in the sense that in many cases, a phenomenon can be faithfully represented by few to several variables and if we can just model these few or several variables, we can model the most part of the phenomenon)

By knowing the characteristics of these two main research step approaches: forward and reverse steps approaches, we can optimise and streamline our research activity so that the effectiveness and efficiency of our research are improved, that is, maximising result impacts while minimising research cost.

Reverse step approach is commonly used for social science research, such as research based on field work or case study [1]. However, nowadays, this reverse step approach is becoming popular for research in natural science and engineering [2].

The increase popularity of reverse step approach for science and engineering field is perhaps due to the abundance of experimental and simulation data, the affordability of powerful computation system and the advanced in machine learning algorithms.

By the end of this post, we will learn about:

The two main types of research approaches
The emphasis on reverse research step approach and when is the best situation to use this approach

Forward research step paradigm: From theory building, hypothesis to experiment or observation

This research step paradigm is commonly used in natural science and engineering research activities.

In general, the steps are started from building or reconstructing a theory or model or hypothesis testing (commonly used in experimental research).

The reconstructed theory or model can be based on extending the previous theory or (rarely) create a new theory from scratch. Extending from the previous theory is for example, adding new terms in a previous model that consider non-linear effects.

Meanwhile, reconstructing hypothesis is very often used for experimental-based research. For example, we hypothesise that the softness of a bread is affected by the amount of material compositions and the temperature homogeneity.

Then, based on this hypothesis, we conduct experiments, collect data and analyse the data to verify whether our hypothesis is accepted or rejected. This analysis is commonly conducted by using statistical testing methods.

The main characteristic of forward research step approach is that this approach is “incremental” in the sense that the research is based on the previous step of mature research or established knowledge. However, this research approach very often can also produce a breakthrough or significant impact or contribution.

The forward research steps are commonly as follows:

1. Getting started

This step is to define a research question or primary construct related to our study. This step is the same for both forward and reverse step paradigm. Another name of this step is problem identification.

2. Designing theory or model

After the important research question or problem is identified, a model or theory or hypothesis is reconstructed. Very often, this activity is based on previous research finding or knowledge. The level of extension from the previous model or theory can be a small advancement or even a breakthrough.

For example, we extend a theory so that the theory also considers additional effects that are not considered in the previous model. Another example is that we build a finite element method (FEM) model based on the previous model built by previous researchers.

3. Designing experimental and/or simulation setup

After a theory or model or hypothesis has been reconstructed, an experimental or simulation setup is prepared. The main goal is to be able to collect data and to verify whether the reconstructed theory or model or hypothesis is accepted or not. “Accepted” means that the model or theory or hypothesis represent the actual data from the experiment and/or simulation.

4. Running experiment and/or simulation

Running experiment and/or simulation is an integral part of a research activity and sometime take the highest amount of time of the total research activity. Very often, preparing an experiment will take a very long time and this activity in itself is iterative to find the most suitable experiment and/or simulation setups and parameters.

5. Analysing data

Analysing data for experiment is usually done by using a statistical method, that is hypothesis testing, due to our data has variations. With statistical method, these variations in our data can be faithfully estimated and our data can be faithfully compared.

For theory or model-based research, the obtained data from experiment and/or simulation are fit with respect to the model predictions. Hence, the model predictions and the experimental data are then compared (using statistical analysis) to decide whether the model or theory agrees with the data or not.

6. Validating theory/model

There is a good principle with regard to our model or theory: all models and theories are wrong or will not represent the actual phenomenon. The most important thing from the comparison of real data and a model is to know how wrong our model is so that we can know the limit of the model, anticipate and use the model correctly.

7. Reaching closure

This is the conclusion step from our research. From this activity we can have a statement how far our model or theory match with real data or whether our initial hypothesis is accepted or rejected.

Numerical Python: Scientific Computing and Data Science Applications - A book Leverage the numerical and mathematical modules in Python and its standard library

Reverse research step paradigm: From theory building, hypothesis to experiment or observation

The common application of this research step is for social science research that are commonly based on field case studies or field observations. This approach focuses on the dynamics that presents in a situation within single settings [1].

In this reverse step, there are similarity on the steps but also there are significant difference with the forward research step paradigm.

Some of the significant differences are especially on the theory dan data comparison and the replacement of theory/model reconstruction to be case study qualitative selection.

The reverse research steps are commonly as follows [1]:

1. Getting started

This step is the same with the forward step approach that is to identify research problems or questions. In this step, a focus should be set to filter out abundance of available options. Sometimes, some knowledge from previous cases can help to identify the correct problems that is worth focusing or pursuing.

2. Selecting cases

This step is neither theory nor hypothesis and it is not based on sampling from population (not based on statistical evidence). Rather, this step is to select a case or phenomena that is “transparently observable”, that is, to select a case or phenomena that is likely to replicate or extend emergent theories.

3. Crafting instruments and protocol

This step commonly combines multiple data collection methods, qualitative and quantitative data analysis and multiple investigators. Qualitative data is important to understand the rationale within quantitative data.

With regard to multiple investigators (such as performing field studies in a team), the benefits are that multiple complementary insights can be gained and various and novel insight can be capitalised.

4. Entering the field or obtaining data/evidence

One of the key ideas of this step is overlapping data collection and analysis, including field notes, flexible and opportunistic data collecting methods.

Field notes are very useful to keep record of what is happening on the field. These notes will be the record of all raw data to be filtered in future. To make a good field note and to extract information from the notes, we should ask critical questions, for example “what am I learning” and “what is the difference between this case and previous cases or studies”

For natural science and engineering, this step can be in a form of multiple data sources with different resolutions or from different measurements.

5. Analysing data

Some of useful method for data analysis in case study of social science research are within-case analysis and cross-case pattern search using divergent techniques.

Within-case analysis is a method to escape from drowning within abundance of collected data or facts. This analysis can be said to be similar to data filtering to find the relevant and important data within the abundance of obtained raw data.

cross-case pattern analysis is driven by the fact that human is a poor processor of information [1]. That is, people always tend to jump to conclusion with limited data, have biases to more “elite” data, ignore basic statistical properties and sometimes ignore data that do not confirm our expectation.

As a results, premature or in the worst-case, a wrong conclusion can be obtained.

The idea of cross-case pattern analysis is to look data in many divergent ways. One of the methods is to look data with within-group similarities and with intergroup differences [1]. Another methods are by grouping cases into different categories and by selecting pairs of cases and listing the similarities and differences between each pair as well as dividing data by data for each source [1].

For natural science and engineering research, common and powerful ways to perform data analysis is to apply data analytic and machine learning algorithms [2].

6. Shaping hypothesis

This is an iterative process to systematically compare emerging frameworks or theories or models from the data observation or evidence to match how well each framework/model fit with the case data.

This shaping hypothesis is a two-part process: refining the definition of a theory or framework and building evidences that measure or compare the theory or framework in each case.

Several methods for this shaping hypothesis are to perform iterative tabulations of evidence for each model/framework, perform replication (not sampling) logic across multiple cases and find “why” behind relationships of data and case studies (qualitative data are very often useful to find “why” relationships).

7. Enfolding literature

This step is to perform comparison of our framework or theory with conflicting literatures and comparison with similar literatures.

Comparison with conflicting literatures is important for some reasons: to estimate the level of confidence with the built framework /theory (many conflicts may lead to the less confidence of the framework or theory) and to trigger more opportunities to refine or extent the theory or framework to reduce the conflicts.

Meanwhile, comparison with similar literatures is important as well for some reasons: to tie together underlying similarities among cases so that the develop framework or theory has a strong validity, wide generalisation and good conceptual level. The implementation for this similar comparison is by taking one or several factors and find these factors within each case.

8. Reaching closure

The developed framework or theory can be considered enough and the iteration steps can be stopped when “theoretical saturation” has been achieved.

“Theoretical saturation” is a point or level at which learning or information gain is minimal in the iteration steps since the learning activities or the learnt information has been seen before in the previous data or observation [1].

It is like ending a manuscript writing when all revisions to be added are minimal. At this point, we can think that the manuscript is ready to be submitted.

Also, the end of the iteration process is also tight with available resources, for example time and funding availability.

Beside a framework or theory, the output of this activity can be in the form of a new concept, such as design (can be a product, system or social engineering design).

Between forward and reverse research step paradigms: The best choice

So, the main question is which research step approach is the best between the two approaches? The answer is there is none!

Both research steps have their own advantages and characteristics. Hence, each step will work best on different research scenarios that suit each of the approach’s characteristics.

For forward research step approach, as being mentioned before, this approach is “incremental” in the sense that it is based on a previous theory or knowledge. Hence, this step is best if our research is extending a previous theory or model (in many cases, this is the most common research activities). Of course, this forward step has been proven to give a breakthrough or significant extension of a previous theory or model.

For reverse research step approach, this approach is best for a research case where very little knowledge is available in the research topic. Very little, if not nothing at all, knowledge has been reported in the research topic to study. In other words, we plan to conduct a research topic from scratch.

Nowadays, this reverse step approach is becoming popular for natural science and engineering research field [2]. Some of the main reasons are that many observation data can be obtained within relatively short period of time or with low cost (thanks to the advancement of measurement and data acquisition instruments), high computing resources can be accessed relatively easy nowadays and advance machine learning algorithms are available alongside robust training (optimisation) algorithms.

Conclusion

Different perspectives on research framework approaches have been discussed. Basically, there are two main approaches: forward (common for science and engineering research) and backward research steps (common for social science research, for example, field study research).

Both approaches are equally important and are iterative. However, each approach has the best used for different research situations.

Forward research step approaches are usually best used when a research activity has already a strong theoretical foundation built by previous research activities. For example, expanding a previously mature theory by considering additional effects that have not been considered before. That is, there are some solid knowledge that have been built and accumulated. This research approach is “incremental” in the sense that it is built on the previous theory and sometimes the “incremental” contribution is a breakthrough or very significant.

Meanwhile, reverse research step approaches are best applied for research activity where very little knowledge are known in a topic or field to research. This research is built based on observations or data. And then, based on these data, a theory is built upon. This research approach is common for social science when a filed case study research activities are a very common practice.

However, nowadays, this research method is gaining popularity for natural science and engineering research fields due to mainly two factors: the abundance of experimental data, the reduction of cost of computation and data collections and the advance of machine learning algorithms.

Reference

[1] Eisenhardt, K.M., 1989. Building theories from case study research. Academy of management review, 14(4), pp.532-550.

[2] Brunton, S.L. and Kutz, J.N., 2022. Data-driven science and engineering: Machine learning, dynamical systems, and control. Cambridge University Press.

You may find some interesting items by shopping here.