Author ORCID Identifier
0009-0004-8527-8948
Document Type
Dissertation
Date of Award
12-31-2024
Degree Name
Doctor of Philosophy in Computing Sciences - (Ph.D.)
Department
Computer Science
First Advisor
Guiling Wang
Second Advisor
Zhi Wei
Third Advisor
Mengnan Du
Fourth Advisor
Ajim Uddin
Fifth Advisor
Wenpeng Yin
Abstract
Large Language Models (LLMs) have emerged as transformative tools across a spectrum of domains, yet their practical deployment reveals a blend of remarkable potential and notable limitations. This research explores innovative methodologies to extend the capabilities of LLMs while addressing critical challenges in their evaluation and application. By leveraging rule-based approaches, the in-context learning capabilities of LLMs, and human-in-the-loop validation across three focused studies, this research introduces robust strategies for dataset synthesis, model enhancement, and model assessment in three distinct domains: natural language processing, financial sentiment analysis, and mathematical reasoning
The first study proposes an efficient data augmentation framework, EASE, tailored for text classification tasks. It integrates rule-based sample augmentation through dependency parsing with transformer-encoder models to generate logically coherent augmented samples, enhancing fine-tuning and evaluation processes. Experimental results demonstrate EASE's superiority over conventional techniques in low-resource settings. The second study designs robust instructional frameworks for human annotators to reduce subjectivity in data annotation for LLM evaluation, particularly in the context of financial sentiment analysis, which often involves complex and ambiguous language. Additionally, it examines how this enhanced contextual interpretation can improve asset price prediction by leveraging LLM-driven sentiment analysis as a predictive feature. The third study 1 focuses on logical reasoning in mathematical problem-solving by integrating LLM capabilities with human-in-the-loop verification to generate a novel evaluation dataset. By presenting LLMs with a series of deliberately flawed mathematical problems, the research highlights the models' limitations in moving beyond mere computation, often hallucinating logical answers instead of genuinely identifying and correcting inherent logical errors.
Collectively, these studies demonstrate the potential of LLMs to push the boundaries of what is computationally possible while also highlighting the crucial need for innovative evaluation strategies to realize their full potential across diverse applications. This work sets new standards for LLM performance and paves the way for their evolution from advanced text generators to sophisticated analytical partners in various fields.
Recommended Citation
Rahman, A M Muntasir, "Pushing the boundaries of large language models: innovations and limitations in nlp, finance, and mathematics" (2024). Dissertations. 1808.
https://digitalcommons.njit.edu/dissertations/1808
