layoutlmv3 finetuned funsd

Introduction

LAYOUTLMV3-FINETUNED-FUNSD is a fine-tuned model based on Microsoft's LayoutLMv3 architecture. It is specifically trained for token classification tasks using the FUNSD dataset.

Architecture

The model is derived from microsoft/layoutlmv3-base, leveraging the LayoutLMv3 architecture that is designed for document understanding tasks, incorporating both textual and layout information.

Training

The model was trained with the following hyperparameters:

• Learning Rate: 1e-05
• Train Batch Size: 16
• Eval Batch Size: 16
• Seed: 42
• Optimizer: Adam with betas (0.9, 0.999) and epsilon 1e-08
• LR Scheduler Type: Linear
• Training Steps: 1000

Training Results

The training and evaluation process yielded the following metrics:

• Loss: 1.1164
• Precision: 0.9026
• Recall: 0.913
• F1 Score: 0.9078
• Accuracy: 0.8330

The training script is accessible at the GitHub repository.

Guide: Running Locally

1. Install Dependencies: Ensure you have the required packages installed, including Transformers, PyTorch, and Datasets.
2. Clone the Repository: Retrieve the model and scripts from the repository.
3. Load the Model: Use the Hugging Face Transformers library to load the model with the from_pretrained method.
4. Prepare the Data: Ensure your dataset is formatted correctly and ready for the token classification task.
5. Run Inference: Execute the model on your dataset to perform token classification.

Suggested Cloud GPUs

To efficiently train or fine-tune this model, consider using cloud-based GPU services such as AWS EC2 with NVIDIA GPUs, Google Cloud Platform with TPU support, or Azure's Machine Learning services.

License

The model and its associated code are distributed under the Apache 2.0 License, which allows for both personal and commercial use, distribution, and modification.