Meet our team at BioTechX Europe in Basel on the 9-10 October 2024, booth 724. Schedule a meeting with our team HERE.
was successfully added to your cart.

    Configuring Automated Backups and Model Training Resources in the Annotation Lab

    Avatar photo
    Ph.D. in Computer Science – Head of Product

    A new generation of the NLP Lab is now available: the Generative AI Lab. Check details here https://www.johnsnowlabs.com/nlp-lab/

    Two new tabs have been added to the Settings page to ease the infrastructure definition for the prediction and training tasks and for defining backup schedules.

    Resource allocation for Training and Preannotation

    Since release 2.8.0, Annotation Lab gives users the ability to change the configuration for the training and preannotation processes. This is done from the Settings page > Infrastructure tab. The settings can be edited by admin users and they are read-only for the other users. The Infrastructure tab consists of three sections named Training Resources, Prenotation Server Resources, Prenotation Pipeline Resources.

    Resources Inclusion:

    1. Memory Limit – Represents the maximum memory size to allocate for the training/preannotation processes.
    2. CPU Limit – Specifies this maximum number of CPUs to use by the training/preannotation server.
    3. Spark Drive Memory – Defines the memory allocated for the Spark driver.
    4. Spark Kry Buff Max – Specifies the maximum memory size to allocate for the Kryo serialization buffer.
    5. Spark Driver Maximum Result Size – Represents the total size of the serialized results of all the partitions for spark.

    NOTE: If the specified configurations exceed the available resources, the server will not start.

    Backup settings in UI

    In this release, AnnotationLab adds support for defining database and files backups via the UI. Any user with the admin role can view and edit the backup settings under the Settings tab. Users can select different backup periods and can specify a target S3 bucket for storing the backup files. New backups will be automatically generated and saved to the S3 bucket following the defined schedule.

    Stay tuned for more exciting features!

    How useful was this post?

    Try The Generative AI Lab - No-Code Platform For Model Tuning & Validation

    See in action
    Avatar photo
    Ph.D. in Computer Science – Head of Product
    Our additional expert:
    Dia Trambitas is a computer scientist with a rich background in Natural Language Processing. She has a Ph.D. in Semantic Web from the University of Grenoble, France, where she worked on ways of describing spatial and temporal data using OWL ontologies and reasoning based on semantic annotations. She then changed her interest to text processing and data extraction from unstructured documents, a subject she has been working on for the last 10 years. She has a rich experience working with different annotation tools and leading document classification and NER extraction projects in verticals such as Finance, Investment, Banking, and Healthcare.

    Annotating Multi-Page Documents Efficiently with Dynamic Pagination and Cross-Page Annotation

    A new generation of the NLP Lab is now available: the Generative AI Lab. Check details here https://www.johnsnowlabs.com/nlp-lab/ Dynamic pagination The support...
    preloader