A Survey of Large Language Model-Based

Generative AI for Text-to-SQL: Benchmarks,

Applications, Use Cases, and Challenges
Aditi Singh1 , Akash Shetty1 , Abul Ehtesham2 , Saket Kumar3 , Tala Talaei Khoei4
1
Department of Computer Science, Cleveland State University, USA
2
The Davey Tree Expert Company, USA
3
The Mathworks, USA
4
Khoury College of Computer Science, Roux Institute at Northeastern University, USA
[email protected], [email protected], [email protected],
arXiv:2412.05208v1 [cs.AI] 6 Dec 2024

[email protected], [email protected]

Abstract—Text-to-SQL systems facilitate smooth interaction of deep learning and neural network models marked a signifi-
with databases by translating natural language queries into cant shift, introducing sequence-to-sequence architectures that
Structured Query Language (SQL), bridging the gap between improved the translation of natural language to SQL [2].
non-technical users and complex database management systems.
This survey provides a comprehensive overview of the evolution The integration of large pre-trained language models
of AI-driven text-to-SQL systems, highlighting their foundational (PLMs) and large language models has further advanced the
components, advancements in large language model (LLM) field [3], enhancing the understanding of natural language
architectures, and the critical role of datasets such as Spider, semantics and the generation of accurate SQL queries. Recent
WikiSQL, and CoSQL in driving progress. We examine the appli- surveys have highlighted the impact of PLMs on text-to-
cations of text-to-SQL in domains like healthcare, education, and
finance, emphasizing their transformative potential for improving SQL parsing, noting their ability to capture complex linguistic
data accessibility. Additionally, we analyze persistent challenges, patterns and improve performance across benchmarks [4].
including domain generalization, query optimization, support for Despite these advancements, challenges remain, particularly
multi-turn conversational interactions, and the limited availabil- in handling complex and cross-domain queries. The develop-
ity of datasets tailored for NoSQL databases and dynamic real- ment of large-scale, human-labeled datasets, such as Spider,
world scenarios. To address these challenges, we outline future
research directions, such as extending text-to-SQL capabilities has been instrumental in evaluating and advancing text-to-SQL
to support NoSQL databases, designing datasets for dynamic systems. These datasets provide diverse and complex queries
multi-turn interactions, and optimizing systems for real-world that test the robustness and adaptability of current models. [1].
scalability and robustness. By surveying current advancements This survey aims to provide a comprehensive overview
and identifying key gaps, this paper aims to guide the next of the evolution of text-to-SQL systems, emphasizing the
generation of research and applications in LLM-based text-to-
SQL systems. integration of artificial intelligence methodologies. We explore
Index Terms—LLM, text-to-SQL, natural language processing, foundational concepts, current benchmarks, datasets, and mod-
artificial intelligence, Gen AI, benchmarks, data sets, schema els, offering insights into the advancements and challenges in
linking, sql generation. the field. By examining the trajectory of text-to-SQL research,
we aim to highlight the progress made and identify areas for
I. I NTRODUCTION future exploration.

The task of translating natural language questions into II. N EED FOR T EXT- TO -SQL
Structured Query Language (SQL) statements, known as text- Text-to-SQL systems provide a specialized solution for
to-SQL, has garnered significant attention within the fields of translating natural language queries into precise SQL state-
natural language processing and database management. This ments, allowing users to interact directly with databases
capability democratizes data access and analysis, enabling without requiring expertise in SQL syntax. While general-
users to interact with databases without requiring in-depth purpose AI models like ChatGPT can assist in generating SQL
knowledge of query languages. The development of AI-driven queries, they often lack the domain-specific optimizations and
text-to-SQL systems has been critical in achieving this goal. accuracy that dedicated text-to-SQL systems are designed to
[1] offer. These systems are tailored to manage complex database
Early approaches to text-to-SQL relied heavily on rule- schemas and ensure the generation of syntactically correct and
based systems and semantic parsing techniques. These meth- efficient SQL queries, significantly enhancing data retrieval
ods, while foundational, often struggled with the diversity and and analysis processes. By focusing exclusively on the task
complexity inherent in natural language queries. The advent of converting natural language to SQL, text-to-SQL systems
 Û Keyword 3) Semantic Parsing: This step involves converting the
Searching natural language query into an intermediate logical form
that represents its meaning. Semantic parsing serves as
j Exploration
a bridge between the user’s intent and the formal SQL
of Models
query.
õ Dataset Analysis 4) SQL Generation: The final component translates the
intermediate logical form into a syntactically correct and
£ Evaluation
executable SQL statement. This requires understanding
Metrics SQL syntax and ensuring that the generated query aligns
with the database schema.
² Applications Advancements in artificial intelligence, particularly in deep
learning and natural language processing, have significantly
 Challenges & enhanced the performance of text-to-SQL systems. For in-
Future Directions stance, the integration of large pre-trained language models has
improved the systems’ ability to understand complex queries
Fig. 1. Methodology for conducting the survey of Text-to-SQL systems.
and generate accurate SQL statements [4].
Despite these advancements, challenges remain, especially
deliver more reliable and contextually appropriate results. This in handling complex and cross-domain queries. Ongoing re-
makes them invaluable in scenarios where precise data manip- search focuses on improving the robustness and adaptability
ulation is critical, such as in healthcare, finance, and business of text-to-SQL systems to address these challenges [2].
intelligence. Furthermore, the development of such systems IV. C URRENT B ENCHMARKS , M ODELS , AND DATASETS
incorporates advancements in natural language understanding,
Evaluating text-to-SQL systems necessitates robust bench-
database schema modeling, and semantic parsing, contributing
marks and datasets. Notable among these are:
to their robustness and usability across diverse application
domains. A. Benchmarks Datasets
• Spider: A large-scale, complex, and cross-domain text-
III. F OUNDATIONS OF T EXT- TO -SQL
to-SQL dataset designed to evaluate the generalization
Text-to-SQL systems are designed to translate natural lan- capabilities of models across different databases and
guage queries into Structured Query Language (SQL) state- query structures [1].
ments, enabling users to interact with databases without re- • Spider 2.0: An advanced evaluation framework featur-
quiring expertise in SQL syntax. The foundational components ing 632 real-world text-to-SQL workflow problems from
of these systems as shown in Figure 2 include: enterprise databases. These databases, often hosted on
platforms like BigQuery and Snowflake, include over
1,000 columns. Spider 2.0 challenges models with com-
plex tasks requiring interaction with SQL workflows,
reasoning over extensive contexts, and generating multi-
query SQL operations exceeding 100 lines, making it
essential for assessing language models in enterprise
scenarios [6].
• WikiSQL: Comprising over 80,000 natural language
questions and corresponding SQL queries, this dataset
is derived from Wikipedia tables and focuses on simple
Fig. 2. Text-to-SQL Process Overview SQL queries [7].
• BIRD (BIg Bench for LaRge-Scale Database
1) Natural Language Understanding (NLU): This in- Grounded Text-to-SQL Evaluation): A comprehensive
volves parsing and interpreting the user’s query to com- dataset containing 12,751 question-SQL pairs across 95
prehend its intent and semantics. Techniques such as databases, totaling 33.4 GB. It spans over 37 professional
tokenization, part-of-speech tagging, and syntactic pars- domains, including blockchain, hockey, healthcare, and
ing are employed to analyze the structure and meaning education, emphasizing challenges such as handling ex-
of the input. tensive database contents and integrating external knowl-
2) Schema Linking: This process connects elements of the edge [8].
natural language query to the corresponding components • CSpider: A Chinese large-scale, complex, cross-domain
in the database schema, such as tables and columns. Ef- text-to-SQL dataset, translated from the original Spider
fective schema linking is crucial for accurately mapping dataset. It comprises 10,181 questions and 5,693 unique
user intents to database structures. SQL queries across 200 databases, aiming to facilitate the
 Fig. 3. LLM Framework for Text-to-SQL [5]

development of natural language interfaces for Chinese language questions, leading to improved performance on
databases [9]. complex queries [14].
• UNITE: A unified benchmark composed of 18 pub- • T5-3B: A transformer-based model fine-tuned on text-
licly available text-to-SQL datasets, encompassing natural to-SQL tasks, demonstrating significant improvements in
language questions from more than 12 domains, SQL generating accurate SQL queries [15].
queries from over 3,900 patterns, and 29,000 databases. • MedT5SQL: Tailored for healthcare, MedTS generates
It introduces approximately 120,000 additional examples SQL queries for patient records using a BERT-based
and a threefold increase in SQL patterns compared to the encoder and LSTM decoder trained on the MIMICSQL
Spider benchmark [10]. dataset. [16].
• CoSQL: The CoSQL dataset is a dialogue-based bench- • EDU-T5: Optimized for educational data, EDU-T5 trans-
mark designed for multi-turn text-to-SQL interactions. lates academic queries into SQL, using a T5-based model
It comprises over 30,000 turns and more than 10,000 with cross-attention mechanisms. [15].
annotated SQL queries, collected from 3,000 dialogues • SQLova: Built on WikiSQL, SQLova generates high-
across 200 complex databases spanning 138 domains. precision general-purpose SQL queries by combining a
Unlike static text-to-SQL datasets, CoSQL emphasizes BERT-based encoder and column attention. [17].
natural conversational interactions, simulating real-world • RAT-SQL: Trained on WikiSQL and Spider, RAT-SQL
scenarios where users refine, clarify, and expand their uses a relation-aware transformer with schema encoding
queries. This makes CoSQL a critical resource for ad- to manage complex multi-table queries. [18].
vancing dialogue-based database interfaces. [11]. • X-SQL: Enhances schema representation by integrating
contextual outputs from BERT-style models, achieving
B. Models state-of-the-art performance on the WikiSQL dataset.
[19].
The progression of text-to-SQL models has been marked by • EHRSQL: A benchmark designed for generating SQL
several key developments (Table I): queries from electronic health records, emphasizing
• Seq2SQL: An early model that employs a sequence-to- domain-specific challenges and evaluation [20].
sequence approach with reinforcement learning to gener- • RASAT: A relation-aware self-attention transformer
ate SQL queries from natural language [7]. model optimized for complex queries and integrated with
• SQLNet: Introduces a sketch-based approach to predict dialogue-based datasets like CoSQL [21].
the SQL query structure before filling in specific details, • PICARD: Parsing incrementally for constrained auto-
improving accuracy and efficiency [12]. regressive decoding, PICARD improves the performance
• TypeSQL: Enhances SQLNet by incorporating type in- of language models like T5-3B on dialogue-based and
formation, enabling the model to handle more complex multi-turn SQL generation tasks [22].
queries involving different data types [13].
• IRNet: Utilizes a graph-based encoder to capture the rela-
tionships between database schema elements and natural
 TABLE I
C OMPARISON OF T EXT- TO -SQL M ODELS

Model Name Dataset Training Method Accuracy

Seq2SQL WikiSQL Seq-to-Seq with Reinforcement Learning 59.4%
SQLNet WikiSQL Sketch-Based with Column Attention 63.2%
TypeSQL WikiSQL Type-Aware Neural Network 82.6%
IRNet Spider Graph Encoder + Intermediate Representation 61.9%
T5-3B Spider, CoSQL Fine-Tuned Transformer 70.0%
PICARD + T5-3B CoSQL Constrained Decoding for Dialogue-Based SQL High
Generation
RASAT+PICARD CoSQL Relation-Aware Self-Attention-augmented T5 with 37.4% IEX
Incremental Parsing
MedT5SQL MIMICSQL BERT-based Encoder with LSTM Decoder for SQL High Accuracy in Medical
Translation Query Translation
EDU-T5 Custom Educational Dataset Fine-tuned T5 Model with Cross-Attention for SQL Optimized
Query Generation
RAT-SQL WikiSQL, Spider Relation-Aware Transformer 69.7%
SQLova WikiSQL BERT + Column Attention 95%
X-SQL WikiSQL BERT-style pre-training with context 91.8%
EHRSQL EHRSQL Benchmark Benchmark Model for EHRs N/A

C. Evaluation Metrics the transformative potential of these systems in addressing

industry-specific challenges.
The performance of Text-to-SQL systems is assessed using
several evaluation metrics that capture different aspects of A. Healthcare and Medical Records - Data Management
query generation quality and system usability: • Clinical Decision Support: Text-to-SQL systems assist
• Exact Set Match Accuracy: Measures the proportion of healthcare professionals in querying patient records, re-
SQL queries that match the ground truth exactly, ensuring trieving relevant medical histories, or aggregating patient
structural correctness [23]. data for epidemiological studies.
• Execution Accuracy: Evaluates the system’s ability to • Patient Record Management: Allows healthcare
generate executable queries by checking if the generated providers to retrieve complex patient information, such as
query returns the correct results when executed on the identifying all patients with certain conditions or recent
database [23]. test results, enabling faster access to critical data.
• Question Match Accuracy: Assesses how well the gen- • Medical Research: Facilitates large-scale data retrieval
erated query corresponds to the natural language question from medical databases for research on disease patterns,
[23]. drug efficacy, or population health studies.
• Interaction Match Accuracy: Specifically for multi-turn
datasets, measures the system’s ability to maintain context B. Educational Tool
and generate coherent queries across dialogue turns [23]. • Adaptive Learning Systems: Text-to-SQL can be in-
These metrics provide insights into the accuracy, robustness, tegrated into educational platforms to create adaptive
and practical usability of Text-to-SQL systems across different learning tools that personalize content based on student
scenarios. data stored in databases.
• Academic Analytics: Enables querying of student perfor-
V. A PPLICATION AND U SE C ASES mance data, helping in curriculum assessment, identifying
at-risk students, and evaluating the impact of educational
Text-to-SQL systems serve as a pivotal tool across di- interventions.
verse industries, enabling natural language interaction with • Student Self-Service: Allows students to query databases
databases to facilitate data retrieval, analysis, and decision- for information on course requirements, academic
making. These systems simplify complex data queries, em- records, or library resources without needing technical
powering non-technical users to unlock valuable insights from expertise.
structured data. Table II summarizes the practical applications,
challenges, and benefits of Text-to-SQL systems across key C. Finance and Banking
domains. Below, we detail specific applications in health- • Financial Reporting and Analytics: Enables financial
care, education, finance, and business intelligence, showcasing analysts and executives to interact with databases for
 Early Models Seq2SQL [7]

SQLNet [12]

TypeSQL [13]
Neural Network Models
IRNet [14]

T5-3B [15]

MedT5SQL [16]
Text-to-SQL Models
Domain-Specific Models EDU-T5 [15]

EHRSQL [20]

SQLova [17]

Advanced Models RASAT [21]

PICARD [22]

RAT-SQL [18]
Relation-Aware Models
X-SQL [19]

Fig. 4. Hierarchical Tree of Text-to-SQL Models

TABLE II
A PPLICATIONS , C HALLENGES , AND B ENEFITS OF T EXT- TO -SQL

Applications Challenges Benefits of Text-to-SQL

• Complex schemas involving multiple intercon-
nected tables for patient, treatment, and diagnosis • Simplifies access to patient records, aiding clini-
data. cians and healthcare staff.
• Domain-specific terminology and abbreviations re- • Improves decision-making through faster data re-
Healthcare quiring specialized understanding. trieval for evidence-based practices.
• Integration of external medical knowledge (e.g., • Automates administrative tasks, reducing workload
guidelines, disease ontologies) for accurate re- and errors.
sponses.

• Diverse data formats across institutions, such as • Facilitates analytics on academic performance, aid-
course catalogs and grade records. ing in identifying strengths and weaknesses.
• Ambiguity in queries due to varying terminologies • Supports personalized learning by analyzing indi-
Education among educators and students. vidual student progress.
• Adapting to multiple educational levels and do- • Scales efficiently to handle large datasets, such as
mains (e.g., K-12 vs. higher education). nationwide assessments.

• High query complexity due to multi-faceted finan- • Enhances fraud detection by enabling effective
cial transactions. querying of transaction data.
• Ambiguity in fraud detection rules and terminology • Provides real-time insights for financial reporting
Finance inconsistencies across organizations. and decision-making.
• Need for efficient query execution in real-time • Improves risk management by analyzing transac-
analytics scenarios. tional and market data effectively.

• Enables universal SQL query generation across

• Lack of generalization across diverse domains and
multiple domains.
transfer learning capabilities.
• Improves accessibility for non-technical users to
• Interpretability issues in generated queries.
General Applications interact with databases.
• Debugging challenges when queries yield unex-
• Enhances accuracy and efficiency in data retrieval
pected results or errors.
tasks.
 generating real-time financial reports, trend analyses, and TABLE III
risk assessments. C OMPARISON OF SQL AND N O SQL DATASET AVAILABILITY
• Customer Service and Query Resolution: Empowers Category Available Datasets
customer service agents to query customer data effi- Relational SQL Spider, Spider 2.0, WikiSQL,
ciently, providing personalized and accurate information CoSQL, BIRD
in real-time. NoSQL None (gap to be addressed)
• Fraud Detection and Prevention: Allows for rapid Dialogue-Based SQL CoSQL
query formulation to monitor transactions for patterns in- Domain-Specific SQL MIMICSQL (Healthcare), EDU-T5
(Education)
dicative of fraud, ensuring timely responses to suspicious
activities.
D. Business Intelligence and Analytics C. Handling Ambiguity in Natural Language
• Market Analysis and Trend Detection: Text-to-SQL • Challenge: Natural language queries can be ambiguous
streamlines data retrieval for analyzing customer behav- or lack specific details required for accurate SQL gener-
ior, tracking sales performance, and identifying emerging ation (e.g., missing parameters, vague terms).
market trends. • Future Direction: Implementing advanced disambigua-
• Inventory and Supply Chain Management: Assists tion techniques or interactive clarification processes,
businesses in monitoring inventory levels, supply chain where the model seeks clarification for ambiguous inputs
metrics, and order statuses to improve operational effi- to ensure query accuracy.
ciency.
• Employee Productivity and HR Analytics: Enables D. Incorporating External Knowledge
querying workforce management data, allowing compa- • Challenge: Domain-specific queries often require contex-
nies to monitor employee productivity, analyze turnover tual knowledge not present in the database schema, such
rates, or assess training needs. as industry-specific terms or common data patterns.
VI. C HALLENGES AND F UTURE D IRECTIONS • Future Direction: Integrating external knowledge bases
or ontologies to enhance the model’s contextual under-
Despite significant advancements, several challenges remain
standing and allow it to interpret queries with a broader
in the development of text-to-SQL systems. Future research
scope.
aims to address these challenges through the following direc-
tions. E. Optimizing SQL Efficiency
A. Industry-Specific Challenges and Solutions • Challenge: While accuracy is essential, the efficiency
Healthcare: of SQL queries is also important, especially for large
• Handling highly sensitive patient data requires robust databases where complex queries can be resource-
privacy-preserving techniques in text-to-SQL systems. intensive.
• Queries often require contextual medical knowledge not • Future Direction: Researching optimization techniques
present in the database schema. for SQL generation, such as index-aware querying or
Finance: using machine learning to predict and avoid performance
bottlenecks in query structures.
• The prevalence of NoSQL databases adds complexity to
adapting text-to-SQL systems. F. Human-in-the-Loop Systems
• Ambiguities in financial terminologies, such as risk-
• Challenge: Fully automated systems may not meet the
related queries, require domain-specific language models.
nuanced requirements of real-world applications.
Education:
• Future Direction: Developing interactive text-to-SQL
• Diverse grading structures and course formats lead to
systems where users can validate or edit SQL outputs,
variability in database schema, complicating schema link- providing an extra layer of accuracy and flexibility for
ing. complex queries.
• Lack of standardized datasets reflecting academic systems
globally limits model generalization. G. Improved Interpretability and Debugging
B. Generalization Across Domains • Challenge: Debugging and interpreting complex SQL
• Challenge: Many text-to-SQL models struggle with do- statements generated by AI models can be challenging,
main adaptation, particularly when faced with unfamiliar particularly in mission-critical applications.
database schemas or industries. • Future Direction: Focusing on interpretability in model
• Future Direction: Developing universal or multi-domain design, enabling end-users to understand how the gener-
models that can interpret and generate accurate SQL for ated SQL queries relate to the original natural language
varied domains without additional retraining or extensive inputs, potentially through visual representations or ex-
customization. planations.
H. Extending Text-to-SQL to NoSQL Databases [10] W. Lan, Z. Wang, A. Chauhan et al., “Unite: A unified benchmark for
text-to-sql evaluation,” arXiv preprint arXiv:2305.16265, 2023.
NoSQL databases, such as MongoDB, Cassandra, and Re- [11] T. Yu, R. Zhang, H. Y. Er, S. Li, E. Xue, B. Pang, X. V. Lin, Y. C.
dis, are crucial for handling unstructured, semi-structured, or Tan, T. Shi, Z. Li, Y. Jiang, M. Yasunaga, S. Shim, T. Chen, A. R.
rapidly changing data. Their flexible schema and distributed Fabbri, Z. Li, L. Chen, Y. Zhang, S. Dixit, V. Zhang, C. Xiong,
R. Socher, W. S. Lasecki, and D. R. Radev, “Cosql: A conversational
architecture make them essential for industries like finance, text-to-sql challenge towards cross-domain natural language interfaces
healthcare, e-commerce, and social networks, where scalability to databases,” ArXiv, vol. abs/1909.05378, 2019. [Online]. Available:
and real-time analytics are critical. https://api.semanticscholar.org/CorpusID:202565697
[12] X. Xu, C. Liu, and D. Song, “Sqlnet: Generating structured queries
Despite advancements in Text-to-SQL systems for relational from natural language without reinforcement learning,” in International
databases, there is a lack of models and datasets designed to Conference on Learning Representations, 2018.
query NoSQL systems, as shown in Table III. [13] T. Yu, Z. Yao, Z. Yang et al., “Typesql: Knowledge-based type-aware
neural text-to-sql generation,” in Proceedings of NAACL, 2018, pp. 588–
• Challenge: Existing benchmarks and models focus on 594.
SQL’s structured schemas, while NoSQL databases re- [14] J. Guo, Z. Zhan, Y. Gao, Y. Xiao, J.-G. Lou, T. Liu, and D. Zhang,
“Towards complex text-to-sql in cross-domain database with interme-
quire support for dynamic, document-based schemas. diate representation,” in Proceedings of the 57th Annual Meeting of
• Potential Solution: Developing datasets tailored the Association for Computational Linguistics (ACL). Association for
to NoSQL with JSON-like structures and models Computational Linguistics, 2019.
[15] C. Raffel, N. Shazeer, A. Roberts et al., “Exploring the limits of transfer
leveraging LLMs for unstructured data analytics [24]. learning with a unified text-to-text transformer,” Journal of Machine
This approach could bridge the gap between structured Learning Research, vol. 21, pp. 1–67, 2020.
and unstructured data paradigms. [16] A. Marshan, A. N. Almutairi, A. Ioannou, D. Bell, A. Monaghan, and
M. Arzoky, “Medt5sql: a transformers-based large language model for
text-to-sql conversion in the healthcare domain,” Frontiers in Big Data,
VII. C ONCLUSION vol. 7, p. 1371680, 2024.
[17] W. Hwang, J. Yim, S. Park, and M. Seo, “A comprehensive exploration
This paper concludes by highlighting the transformative po- on wikisql with table-aware word contextualization,” 2019. [Online].
tential of text-to-SQL systems in bridging the gap between nat- Available: https://arxiv.org/abs/1902.01069
[18] B. Wang, R. Shin, X. Liu, O. Polozov, and M. Richardson, “Rat-sql:
ural language queries and database interactions, empowering Relation-aware schema encoding and linking for text-to-sql parsers,”
non-technical users across various domains. While significant 2021. [Online]. Available: https://arxiv.org/abs/1911.04942
advancements have been achieved through AI-driven models [19] P. He, Y. Mao, K. Chakrabarti, and W. Chen, “X-sql: reinforce
schema representation with context,” 2019. [Online]. Available:
and extensive benchmarks, critical challenges such as handling https://arxiv.org/abs/1908.08113
domain-specific complexities, extending support to NoSQL [20] G. Lee, H. Hwang, S. Bae, Y. Kwon, W. Shin, S. Yang, M. Seo,
databases, and improving query efficiency remain. The survey J.-Y. Kim, and E. Choi, “Ehrsql: A practical text-to-sql benchmark
for electronic health records,” arXiv preprint arXiv:2301.07695, 2023.
underscores the need for future research to develop specialized [Online]. Available: https://arxiv.org/abs/2301.07695
datasets, enhance model generalization, and integrate contex- [21] J. Qi, J. Tang, Z. He, X. Wan, Y. Cheng, C. Zhou, X. Wang,
tual knowledge to address these gaps, thereby expanding the Q. Zhang, and Z. Lin, “Rasat: Integrating relational structures into
pretrained seq2seq model for text-to-sql,” 2022. [Online]. Available:
scope and utility of text-to-database technologies. https://arxiv.org/abs/2205.06983
[22] T. Scholak, N. Schucher, and D. Bahdanau, “Picard: Parsing
R EFERENCES incrementally for constrained auto-regressive decoding from language
[1] T. Yu, R. Zhang, K. Yang et al., “Spider: A large-scale human-labeled models,” 2021. [Online]. Available: https://arxiv.org/abs/2109.05093
dataset for complex and cross-domain semantic parsing and text-to-sql [23] B. G. Ascoli, Y. S. R. Kandikonda, and J. D. Choi, “Esm+:
task,” in Proceedings of the 2018 Conference on Empirical Methods in Modern insights into perspective on text-to-sql evaluation in the
Natural Language Processing, 2018, pp. 3911–3921. age of large language models,” 2024. [Online]. Available: https:
[2] L. Shi, Z. Tang, N. Zhang et al., “A survey on employing large language //arxiv.org/abs/2407.07313
models for text-to-sql tasks,” arXiv preprint arXiv:2407.15186, 2024. [24] H. Dai, B. Y. Wang, X. Wan, B. Dai, S. Yang, A. Nova, P. Yin,
[3] A. Singh, “Exploring language models: A comprehensive survey and P. M. Phothilimthana, C. Sutton, and D. Schuurmans, “Uqe: A
analysis,” in 2023 International Conference on Research Methodologies query engine for unstructured databases,” 2024. [Online]. Available:
in Knowledge Management, Artificial Intelligence and Telecommunica- https://arxiv.org/abs/2407.09522
tion Engineering (RMKMATE), 2023, pp. 1–4.
[4] Z. Hong, Z. Yuan, Q. Zhang et al., “Next-generation database interfaces:
A survey of llm-based text-to-sql,” arXiv preprint arXiv:2406.08426,
2024.
[5] L. Shi, Z. Tang, N. Zhang, X. Zhang, and Z. Yang, “A survey on
employing large language models for text-to-sql tasks,” 2024. [Online].
Available: https://arxiv.org/abs/2407.15186
[6] F. Lei, J. Chen, Y. Ye, R. Cao, D. Shin, H. Su, Z. Suo, H. Gao, W. Hu,
P. Yin, V. Zhong, C. Xiong, R. Sun, Q. Liu, S. Wang, and T. Yu, “Spider
2.0: Evaluating language models on real-world enterprise text-to-sql
workflows,” 2024. [Online]. Available: https://arxiv.org/abs/2411.07763
[7] V. Zhong, C. Xiong, and R. Socher, “Seq2sql: Generating structured
queries from natural language using reinforcement learning,” arXiv
preprint arXiv:1709.00103, 2017.
[8] F. Li, H. Yu, X. Li et al., “Can llm already serve as a database
interface? a big bench for large-scale database grounded text-to-sqls,”
arXiv preprint arXiv:2305.03111, 2023.
[9] Q. Min, Y. Shi, and Y. Zhang, “A pilot study for chinese sql semantic
parsing,” 2019. [Online]. Available: https://arxiv.org/abs/1909.13293