Insights into the nutritional prevention of macular degeneration based on a comparative topic modeling approach

Background and significance

Very rarely in science is consensus derived from a single report. Making sense of the “evidence landscape” on any research question often requires synthesizing results across multiple studies to elucidate the prevailing direction of known facts. Two main types of evidence synthesis are standard in science. One is meta-analysis (MA), the statistical combination of results across multiple studies. The second is systematic review (SR), the collation of relevant evidence from a sample of studies (without performing a statistical unification). Natural language processing (Chowdhary, 2020) (NLP) is the field dealing with the computational understanding (and, more recently, generation) of text and speech. NLP can support MA and SR by automating literature searches and fact/figure extraction from published studies.

Report aggregation isn’t only valuable in the context of peer-reviewed studies. Unstructured anecdotal reports have also been useful in validating research directions across domains, for instance, regarding the potential therapeutic properties of cannabinoids for central nervous system disorders (Croxford, 2003) and the role of apex predators in sensitive ecosystems (Somaweera et al., 2020). In some cases, NLP has proven helpful in analyzing themes in anecdotal reports, such as mental health outcomes during pandemics and extracting insight from patient reports (Marshall et al., 2022; Van Buchem et al., 2022). In this light, collating reports is sometimes inherent to the discovery phase of new phenomena worth studying, with anecdotal observations serving as directional indicators for further research. Indeed, some domains can only be explored via sophisticated instrumentation and the application of advanced theory. In other domains, however, patient experiences or eyewitness testimony can have a real bearing on unanswered questions. More examples include tracking the benefits/side effects of new treatments, understanding symptoms of emerging pathogen variants, observing the effects of climate change on local ecosystems, studying unidentified phenomena in nature, comprehending the psychological impacts of new technologies, and more.

Relevant literature

Within the scope of this study, the focus will be on published literature rather than anecdotal reports (although the proposed method can theoretically have applications for both). In the context of MA and SR, the application of NLP has arguably fallen into two main discernible subcategories: systematic data extraction and literature search (though not limited to these). Text mining is a subset of NLP dealing with the computational extraction of raw data from text (Hassani et al., 2020), e.g., numerical values, figures, and names. Various examples exist in the literature of capturing information from papers via text mining for performing MA. Mutinda et al. (2022) propose a named entity recognition model to identify participants, intervention, control, and outcomes (PICO) from clinical trials. In another domain, the Neurosynth Corpus (Neurosynth, 2023) is created via text mining, namely by tying keyword mentions to fMRI images from a large sample of studies to assemble a library of neuroimaging representations associated with cognitive, physical, and emotional states. Additionally, text mining is leveraged to gather data for MA to identify xerostomia drug targets (Beckman et al., 2022), evaluate the efficacy of educational software (Costa et al., 2020), and augment biomarker discovery (Bhatnagar et al., 2022).

The other preeminent category is NLP-enabled methods for literature search. Feng, Chiam & Lo (2017) explore the value of NLP for automating parts of the SR process. They find that the study selection phase of SR most commonly benefits from text-mining approaches compared to later stages of SR, such as assessing study quality or other types of screening. Marshall et al. (2018) introduce a freely available framework for searching studies while filtering for randomized controlled trials (RCTs). They achieve this by using the support vector machine (SVM) (Pisner & Schnyer, 2020) and convolutional neural network (CNN) (Albawi, Mohammed & Al-Zawi, 2017) machine learning architectures for document classification as RCT vs. non-RCT. Cohen et al. (2015) introduce a system that uses the Liblinear (Fan et al., 2008) fast linear implementation of the SVM to identify RCTs. O’Mara-Eves et al. (2015) show how NLP can be used to prioritize the order of study retrieval based on the likelihood of topic relevance. In another recent study, Kim & Gil (2019) demonstrate a topic-modeling implementation for study retrieval involving the Term-Frequency Inverse-Document-Frequency algorithm (TF-IDF). Topic modeling can be considered the subset of text mining that deals with identifying the overarching subject matter of documents, with obvious value for literature search and relevant information retrieval. Based on a review of related work, it is clear that NLP and topic modeling offer practical value for key phases of MA and SR.

Various approaches for topic modeling have been widely adopted, including latent Dirichlet allocation (LDA) (Blei, Ng & Jordan, 2003), latent semantic analysis (LSA) (Dumais, 2004), and TF-IDF. Latent Dirichlet allocation is frequencly used for supporting MA and SR (Mo, Kontonatsios & Ananiadou, 2015; Choi, Lee & Sohn, 2017; Asmussen & Møller, 2019). TF-IDF proves to be an indispensable measure that is still applied (either with or without modifications) in a wide range of novel use cases in recently proposed papers (Kim & Gil, 2019; Gomes, da Silva Torres & Côrtes, 2023; Jalilifard et al., 2021; Rawat et al., 2021; Yuan et al., 2020). TF-IDF is therefore still a current topic in the NLP field as indicated by the relative recency of the literature leveraging this algorithm. In NLP terminology, “documents” refers to individual entries (e.g., papers, social media posts, or patient reports), whereas the “corpus” is the broader dataset that is the collection of documents. TF-IDF captures the salient topics of a given document by identifying the most ubiquitous terms within the document that are simultaneously rare throughout the larger corpus. The highest-ranking terms are most likely representative of the document’s subject matter. Intuitively, if “chromosome” appears frequently in a specific document but not in the rest of the corpus, it will have a high score. Conversely, the word “and” will have a low score since it occurs frequently in both the document and across the corpus and does not represent a meaningful topic. While topic modeling methods are abundant in the literature (including for supporting SR and MA, as demonstrated earlier), this article explores a nuanced type of topic modeling not well-explored in the literature, the goal being to yield insights into the prevention of debilitating diseases.

Contributions

NLP methods for MA or SR typically focus on literature search (via topic modeling and classification) or text mining for facts and figures. Instead, the interest of this article is a topic modeling approach for finding differentiating subtopics in reports containing contradictory findings to others on the same question, presenting unique opportunities to yield insight and advance understanding. Seemingly conflicting results can be found on a vast number of questions across medical literature (Lamers et al., 2021), for instance, whether exposure therapy is effective for PTSD (Huang et al., 2022; Markowitz & Fanselow, 2020), the role of meat consumption in a healthy diet (Rohrmann et al., 2013; Leroy & Cofnas, 2019) and many other examples. Indeed, disparities in outcome are often attributable to general methodology, study design, or demographics studied. However, in many cases, specific compounds, dosages, practices, or other discoverable factors may be associated with outcomes consistently enough to warrant further study. Consider resveratrol, a compound studied for various health benefits (including antioxidant and anti-tumor activity). Studies with the most significant results tended to administer resveratrol with liposomes since it is a fat-soluble compound (with limited water solubility), as found in the SR conducted by Amri et al. (2012). This detail (i.e., administration with liposomes) may be responsible for divergent outcomes in early studies examining the benefits of resveratrol before there was considerable literature volume on this topic. Another example is the question of whether nutritional supplementation prevents macular degeneration (MD) in any significant way. MD prevention is chosen as the area of focus since it affects millions of people annually, and its treatment and prevention represents an ongoing medical research avenue. Individual studies may include different nutritional compounds than others. If particular compounds are mentioned more frequently in the context of significant findings for the prevention of MD, those compounds would be of interest for further study as potential causal factors for the desired outcome. This article demonstrates an NLP-enabled approach for finding such factors. Going forward, these factors will simply be called “potential determinants.”

While TF-IDF models topics by using term frequencies within a document and across a corpus (as explained above), a new approach is proposed to identify potential determinants. Unlike most current topic modeling techniques, the proposed approach assumes two corpora, one containing studies showing significant positive effects and another containing studies showing insignificant/negative effects for a relationship of interest (e.g., nutritional compounds and the prevention of MD). These will be called the “positive” and ”negative” corpus for simplicity. Finding potential determinants is not as trivial as finding topics unique to the positive corpus (since some overlap between corpora is expected). At the same time, it isn’t as simple as comparing a topic’s frequency between the corpora; the consistency of a topic’s distribution across documents in the positive corpus also signals its interest as a potential determinant (to ensure a higher frequency is not simply attributable to a single document). Some terms may merely be tangentially mentioned in the report (rather than being directly studied), confounding the matter even more. Further, potential determinants may not necessarily be the most frequently discussed topics in the positive corpus while still having a meaningful association with the outcome of interest. Therefore, while methods exist in the literature for directly comparing topics between datasets (e.g., for news event comparison (Hua et al., 2020)), a specially conceived approach is needed to qualify topics as potential determinants in the context of divergent study results. The proposed method takes some basic intuitions from TF-IDF, generating a score for each term in the positive corpus. However, the proposed approach fundamentally differs from TF-IDF while utilizing the same attributes. For a given term, the proposed topic significance score is a function of its proportional occurrence in the positive corpus (out of all occurrences across both corpora) and its distribution across documents within the positive corpus. Candidate topics are also filtered for membership within a category of interest. The latter criterion ensures that results are focused within a particular class (e.g., nutritional compounds), enabling a specific and meaningful search for potential determinants. While large language models may seem helpful for the same use case, they can produce spurious information in their current form and may have limited explainability (Azamfirei, SR & Fackler, 2023; Krause et al., 2023). An explainable method is of current interest (not to rule out the promise of LLMs for this use case in the future, perhaps in tandem with methods like the proposed approach).

The proposed method will be tested on papers addressing the general question of whether nutritional compounds can prevent the development or progression of MD in a significant way. In other words, it will be applied to research featuring breadth and variety in the compounds it tests/reviews (e.g., multivitamins, diets, or supplement formulations) rather than narrow scope (e.g., one particular compound). The objective of the proposed method is to extract specific potential determinants from the broad-scope reports. Validation will consist of a retrospective literature search on each identified compound. Suppose the results of the proposed method are supportable for benefiting MD (according to narrow-scope follow-up studies that address each compound); this would substantiate the proposed method’s ability to pinpoint relevant topics for desired outcomes from a wide range of candidates, inform new research directions, and support evidence synthesis including MA and SR in the future. Most importantly, insights on the nutritional prevention of MD may be uncovered in the process. If the results prove meaningful, the method can be applied to anecdotal evidence in future work. Doing so at this stage, however, would make the results difficult to validate on questions for which formalized study is lacking. By instead applying the proposed approach to the scientific literature on the topic of nutrition for MD, there is a path to qualifying the results via a follow-up literature search. Results will also be compared with the current state-of-the-art in topic modeling.

The rest of this article is organized as follows. ‘Materials & Methods’ will formalize the definition of TF-IDF, the proposed method, key differences between the proposed method and TF-IDF, the data used, pre-processing methods, experimental structure, and the state-of-the-art baseline implemented to contextualize results. ‘Results’ will detail the results. In ‘Discussion’, an interpretation of the results is provided. Limitations are also discussed. Finally, ‘Conclusions’ summarizes all findings and considers future research directions.

TF-IDF (background introduction)

TF-IDF is an extensively used statistical approach for topic modeling (including recently). As of 2015, roughly 83% of text-based digital recommender systems are thought to rely on TF-IDF (Beel et al., 2015). Therefore, one cannot make the argument that TF-IDF is outdated; it is still used thanks to its efficiency, reliability, and flexibility for topic modeling use cases. TF-IDF is not directly applied in the proposed method, but understanding it is essential for background. The intuition behind TF-IDF is as follows: in a given document, the most topic-significant terms occur frequently in the document while appearing infrequently across the corpus. Note “keywords,” “terms,” and “topics” can be used interchangeably in this context. Tempering a term’s score by its distribution across the corpus accounts for some terms occurring more frequently in general, independently of subject matter. Therefore, a term’s uniqueness to a particular document and its internal frequency within that document is a proxy for its topic relevance. Formally, TF-IDF yields a relative frequency score tf(t, d) for a given term t in the document d according to the following classical definition (Joachims, 1997): (1)t f t , d = f t , d ∗ l o g | D | / D F t

where f(t, d) is the raw number of times the term t appears in document d, |D| is the total number of documents in the corpus, and DF(t) is the number of documents in which the term t occurs at least once. The higher the value of tf(t, d), the greater the likelihood of term t representing a significant topic in the document.

Proposed method (main contribution)

The proposed method in some ways draws from TF-IDF but is not simply concerned with extracting the primary topics from a given document. Instead, the interest is in topics that:

1. Have the highest proportional occurrence in reports of significant results for the outcome of interest (out of total occurrences of the term across all documents).

2. Have the most consistent distribution across reports of significant results for the outcome of interest.

3. Represent “studyable” factors (rather than incidental verbiage).

Such topics are referred to as potential determinants in this article, as stated in ‘Contributions’. As mentioned, the proposed method inherently requires two corpora, one comprising significant results and the other negative or insignificant results on the same general question. These will respectively be referred to as Cp and Cn going forward.

Table 1 introduces the proposed method’s term scoring structure in terms of the same attributes used in TF-IDF below.

A notable difference between TF-IDF and the proposed approach is the use of a term’s document distribution. In TF-IDF, which assumes only one corpus, there is an inverse correlation between a term’s score and its distribution across documents. In the proposed method, however, greater distribution throughout Cp instead boosts a given term’s score, but not throughout Cn.

Outside of scoring, candidate topics are filtered for lexical membership in a list. The list contains candidate terms within a category of interest (e.g., nutritional compounds). Regardless of the score, topics must belong to the list to be included in the final results, adding specificity to the search.

Formally, the proposed approach is defined as follows:

Let DCp represent the number of documents in the corpus Cp: (2)D C p = | D ∈ C p | where D represents documents.

Let DCp(t) denote the number of documents in the corpus Cp in which a given term t occurs at least once: (3)D C p t = | D ∈ C p : t ∈ D |

Let n(t, Cp) represent the raw total count of a given term t’s occurrences across documents in Cp only.

Let n(t, Cp, Cn) denote the raw total count of a term t’s occurrences across all documents, inclusive of Cp and Cn.

Let b(t) represent the proportional occurrence of a given term t specifically in Cp out of all occurrences: (4)b t = n t , C p / n t , C p , C n

Therefore, the score a(t) for a given term t in Cp is defined as (5)a t = 1 2 b t + D C p t / D C p

where possible scores range from 0 to 1. A higher a(t) value signals greater topic importance as a potential determinant. Terms must be filtered for membership in a qualifying list of candidates to search for, simply defined as the list . Filtering must also be performed to ensure the b(t) value is above 0.5. This ensures each candidate term already has greater proportional occurrence in Cp with the distribution serving as a secondary criterion.

Note the proportion of a term’s occurrence in Cp out of all occurrences is used, but its raw count is not (unlike TF-IDF, which merely uses the raw count of a term within a document). This is because potential determinants may not necessarily be the most frequent terms in Cp while still having a unique association with the outcome of interest, particularly if accompanied by considerable distribution throughout Cp. Therefore, its proportional occurrence in Cp is more of interest than its raw count. Simultaneously, using proportional values serves as a means of score normalization. Figures 1, 2, and 3 illustrate various scenarios to elucidate the nuances of the proposed method’s scoring system.

Study selections

The following studies were used to test the proposed method. A careful manual selection was made to satisfy three assumptions for studies in Cp and Cn:

1. Consistent focus on the same objective, namely assessing the impact of nutritional compounds on the development or progression of MD.

2. Broad scope (e.g., the effects of nutritional supplements, diets, or antioxidants on MD) vs. narrow scope (e.g., the effects of beta-carotene on MD).

3. Conclusive outcomes clearly categorized as either significant and positive findings or insignificant/negative findings. Figure 4 is a PRISMA flow diagram of the study selection process.

4. A unique set of entries. Some secondary literature was included. SRs are treated as standalone entries since each comprises a unique set of study selections, interpretations, and conclusions.

To search for relevant articles, the keywords “macular degeneration prevention” and “macular degeneration treatment” were used as base keywords. Each of these base keywords were respectively combined with “nutritional”, “nutrition”, “nutrient”, “nutrients”, “antioxidants”, “antioxidant”, “vitamin, “vitamins”, “multivitamin”, “dietary”, “diet”, “food” and “foods. The search only included journal articles.

Note, there were many on-topic studies that either had too narrow a scope (focusing on a single compound for MD) or findings that were too inconclusive to be suitable for comparative analysis. Within the criteria, all apparent qualifying studies were included to avoid selection bias to the degree possible following an extensive literature search (as of August 2023).

Studies in Cp contained the following (see Table 2):

Studies in Cn comprised the following (see Table 3):

Documents in Cp had a combined length of 64,453 words. Documents in Cn had a combined length of 65,591 words. Note, the latter refers to total word counts, not the unique set of terms.

Data pre-processing and testing

Text normalization was performed according to the following steps applied to each document in Cp and Cn:

1. Lemmatization (Cambridge University Press, 2009) was applied to each term in every document across Cp and Cn. This process refers to the reduction of a term to a base form, e.g., “researched” and “researching” to simply “research.” Without lemmatization, the various forms of a term would be treated as altogether separate topics, confounding results. Lemmatization was performed via Natural Language Toolkit (NLTK Project, 2023a), a library for Python.

2. Removal of stopwords via Natural Language Toolkit’s stopword removal method (NLTK Project, 2023b). Stopwords are a set of the most frequently used terms in English (regardless of subject matter), such as “and,” “it,” or “the.”

3. Reduction of all words to lowercase form.

4. Punctuation removal.

After applying the above pre-processing steps, each document in Cp and Cn was treated as a multiset of normalized terms. Figure 5 demonstrates the text normalization process.

The unique set of terms present across all documents in Cp was used for testing. Formally, the set of test terms B is defined as (6)B = t ∈ D : D ∈ C p

where t represents terms and D represents documents.

For each term t in the set B, an a(t) score was generated as described in ‘Proposed Method (Main Contribution)’.

As specified earlier, topics are filtered for membership within a list q. For this experiment, q contained a lexicon of nutritional compounds to search for. This list was obtained from FooDB (FooDB, 2023), which offers a database of over 3,000 detected and quantified nutritional compounds. FooDB is supported by the Canadian Institutes for Health Research (Government of Canada, 2023). The text normalization steps detailed above were applied to each term obtained from FooDB.

Only the terms with b(t) values above 0.5 were considered. Further, only the terms with final a(t) scores above the 75th percentile were considered for final analysis. Terms with scores in the said range were then filtered for membership in q.

Results were contextualized by performing traditional topic modeling on the documents in Cp. To apply TF-IDF, the following steps were applied for each term identified by the proposed method:

1. For each document in Cp, the said term’s TF-IDF score ranking (if present in the document) was calculated using the internal frequency of the term in that document with respect to its document frequency across Cp. The ranking is handled such that the most prominent topic has a rank of 1, the second most prominent has a rank of 2, et cetera.

2. The same was done for the same term but this time using each document in Cn, while using Cn to establish document frequency.

3. The mean TF-IDF ranking for the term was established both in Cp and Cn, respectively.

To further contextualize results with a more recent baseline, Top2Vec (Angelov, 2020; Karas et al., 2022) was applied to Cp and Cn. Top2Vec uses document and word semantic embeddings (Gutiérrez & Keith, 2019) to automatically identify the salient topics in a group of documents without specifying the number of topics to expect, as is the case with LDA (Blei, Ng & Jordan, 2003). Using a recent model that leverages document and word embeddings is an important step in relating results to the current state-of-the-art. Top2Vec yields groups of terms that are clustered according to their topic association, and has been shown to provide added benefits compared to LDA and TF-IDF. In terms of Top2Vec’s results, each resulting cluster represents a salient topic, and the words they comprise are associated within the said topic. Top2Vec was applied separately to Cp and Cn to compare results between the corpora and determine if the proposed approach introduces any added benefit when applied for comparative topic modeling. Top2Vec was implemented using the pre-trained universal-sentence-encoder (Angelov, 2020). Note, the Top2Vec implementation in the Python library used does not require stopword removal or lemmatization (Python Package Index (PyPI), 2024a). Top2Vec was therefore applied to the raw documents in keeping with the intended use specified in the library documentation.

As another additional baseline, BERTopic (Grootendorst, 2022) was applied to each respective corpus as with Top2Vec. BERTopic generates document embeddings via transformer-based models, clusters these embeddings, and outputs topic representations via a TF-IDF based procedure. BERTopic was implemented using the pre-trained “paraphrase-MiniL3-v2’ model (Python Package Index (PyPI), 2024b).