tapex base

Introduction
TAPEX (Table Pre-training via Execution) is a pre-training approach designed to enhance models with table reasoning capabilities. It achieves this by learning a neural SQL executor over a synthetic corpus of executable SQL queries. TAPEX is based on the BART architecture, which combines a bidirectional encoder and an autoregressive decoder.

Architecture
TAPEX utilizes the BART architecture, a sequence-to-sequence (seq2seq) transformer model. It features a bidirectional encoder akin to BERT and an autoregressive decoder similar to GPT. This design supports the execution of SQL queries on tables by simulating neural SQL execution.

Training
The model is pre-trained on a synthetic corpus that consists of automatically generated executable SQL queries. Although the TAPEX model can execute SQL queries directly, it is primarily intended for fine-tuning on supervised datasets. Tasks suitable for fine-tuning include table question answering and table fact verification.

Guide: Running Locally

1. Install Required Libraries: Ensure you have the transformers library and pandas installed.
2. Load Model and Tokenizer:

   from transformers import TapexTokenizer, BartForConditionalGeneration
   import pandas as pd
   
   tokenizer = TapexTokenizer.from_pretrained("microsoft/tapex-base")
   model = BartForConditionalGeneration.from_pretrained("microsoft/tapex-base")
   

3. Prepare Data: Create a pandas DataFrame with the table data.

   data = {
       "year": [1896, 1900, 1904, 2004, 2008, 2012],
       "city": ["athens", "paris", "st. louis", "athens", "beijing", "london"]
   }
   table = pd.DataFrame.from_dict(data)
   

4. Query Execution: Tokenize the input and generate outputs.

   query = "select year where city = beijing"
   encoding = tokenizer(table=table, query=query, return_tensors="pt")
   outputs = model.generate(**encoding)
   print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
   # Output: ['2008']
   

5. Fine-Tuning: Access the fine-tuning script here.

Suggested Cloud GPUs: Consider using cloud services with GPU capabilities, such as AWS EC2, Azure, or Google Cloud Platform, for efficient model execution and fine-tuning.

License
TAPEX is licensed under the MIT License, allowing for considerable freedom in use, modification, and distribution.