AsposeAIModelConfig
Contents
[
Hide
]AsposeAIModelConfig class
Represents configuration settings for loading an LLM model.
public class AsposeAIModelConfig
Constructors
| Name | Description |
|---|---|
| AsposeAIModelConfig() | The default constructor. |
Properties
| Name | Description |
|---|---|
| AllowAutoDownload { get; set; } | If true, the model will be automatically downloaded if it is not available locally. The default value is false. When FileModelPath is specified, the model is loaded from that file and automatic download is not required. |
| ContextSize { get; set; } | Defines the maximum number of tokens the LLM can use as context during inference. If null, the default context size defined by the model will be used. Larger values allow the model to consider more text but may require more memory. |
| DirectoryModelPath { get; set; } | Optional directory where HuggingFace models will be searched for and cached. Use FileModelPath instead when you want to point directly to a manually downloaded model file. If not set, a default system location will be used. |
| FileModelPath { get; set; } | Local path to the model file. If specified, this file will be used instead of downloading the model. Default empty. |
| GpuLayers { get; set; } | Number of GPU layers to use for the model. If not specified, the default value (40) will be used. Set to 0 to run entirely on the CPU. |
| HuggingFaceFileName { get; set; } | Optional model’s name to use when downloading from HuggingFace. Examples: “TableGPT2-7B.Q4_K_S.gguf”. Default for table AI - TableGPT2-7B.Q4_K_S.gguf. |
| HuggingFaceRepoId { get; set; } | ID of the model on HuggingFace (e.g., “openai/gpt2”). If specified, the model can be loaded from the local cache or downloaded from HuggingFace when AllowAutoDownload is enabled. Default bartowski/Qwen2.5-3B-Instruct-GGUF. |
See Also
- namespace Aspose.OCR.AI
- assembly Aspose.OCR
