Deploying John Snow Labs Medical LLMs on Databricks: Three Flexible Deployment Options

from johnsnowlabs import nlp, medical

# Initialize Spark session with your license
spark = nlp.start()

# Load document assembler
document_assembler = nlp.DocumentAssembler()\
    .setInputCol("text")\
    .setOutputCol("document")

# Load medical LLM
# Recommended: 8B q8 model for A10 GPU (24GB VRAM)
medical_llm = medical.AutoGGUFModel.pretrained("jsl_meds_8b_q8_v4", "en", "clinical/models")\
    .setInputCols("document")\
    .setOutputCol("completions")\
    .setBatchSize(1)\
    .setNPredict(500)\
    .setUseChatTemplate(True)\
    .setTemperature(0)

# Create pipeline
pipeline = nlp.Pipeline(stages=[
    document_assembler,
    medical_llm
])

# Process clinical data
results = pipeline.fit(data).transform(data)

from johnsnowlabs import nlp, medical

# Models are pre-loaded in the container
available_models = [
    "jsl_medm_q8_v3",          # 14B general medical model
    "jsl_meds_8b_q8_v4",       # 8B specialized model
    "jsl_meds_vlm_3b_q8_v1",   # 3B vision model
    "jsl_meds_ner_q8_v4"       # 3.5B entity extraction
]

# Load a vision model for document processing
vlm = medical.AutoGGUFModel.pretrained("jsl_meds_vlm_3b_q8_v1", "en", "clinical/models")\
    .setInputCols("document")\
    .setOutputCol("extracted_data")\
    .setUseChatTemplate(True)\
    .setTemperature(0)

# Prompt for structured extraction
extraction_prompt = """
Extract the following information from this medical document:
1. Patient demographics (name, age, gender)
2. All laboratory test results with values and units
3. Any medications mentioned with dosage and frequency

Return as structured JSON.
"""

# Process document image
vlm_pipeline = nlp.Pipeline(stages=[
    document_assembler,
    vlm.setPrompt(extraction_prompt)
])

# Single image processing: ~5-8 seconds on H100
result = vlm_pipeline.transform(image_data)

# sample output

{
  "demographics": {
    "name": "Patient Name",
    "age": 45,
    "gender": "Male"
  },
  "lab_results": [
    {"test": "Hemoglobin", "value": "14.2", "unit": "g/dL"},
    {"test": "Glucose (fasting)", "value": "105", "unit": "mg/dL"}
  ],
  "medications": [
    {
      "name": "Metformin",
      "dosage": "500mg",
      "frequency": "twice daily"
    }
  ]
}

import requests
import json

# Your private endpoint URL (provided after subscription)
endpoint_url = "https://.databricks.com/serving-endpoints/jsl-medical-llm-32b"

headers = {
    "Authorization": f"Bearer {databricks_token}",
    "Content-Type": "application/json"
}

# Clinical summarization request
payload = {
    "inputs": {
        "prompt": """
        Summarize the following clinical note into a concise assessment:
        
        [Clinical note text here...]
        """,
        "max_tokens": 500,
        "temperature": 0.0
    }
}

# Make synchronous call
response = requests.post(endpoint_url, headers=headers, json=payload)
result = response.json()

print(result["completions"][0]["text"])

{
    "min_instances": 0,  # Scale to zero when idle
    "max_instances": 5,
    "scale_down_delay": "5m"
}
# Pros: Minimal cost during idle periods
# Cons: ~30-60s cold start on first request

{
    "min_instances": 2,  # Always warm
    "max_instances": 20,
    "target_utilization": "60%",
    "scale_down_delay": "15m"
}
# Pros: No cold starts, smooth scaling
# Cons: Higher minimum cost

Choose Healthcare NLP Library if:

• You need to combine LLMs with specialized NER/RE models
• Building complex multi-stage pipelines (e.g., NER → RAG → Summarization)
• Want maximum flexibility in prompt engineering and model chaining
• Already using Spark NLP for other healthcare tasks

Choose Container Services if:

• Standardizing deployments across multiple teams
• Need reproducible environments for compliance/audit
• Prefer Docker-based MLOps workflows
• Want to version-lock your entire stack

Choose Private Endpoints if:

• Processing variable workloads (e.g., batch jobs during specific hours)
• Building microservices or REST APIs consuming LLMs
• Want to minimize DevOps overhead
• Need auto-scaling for unpredictable traffic patterns

Key Takeaways

• John Snow Labs provides the most accurate medical LLMs on the market, outperforming GPT-4o, Claude, and Med-PaLM-2 on healthcare benchmarks by 4–11 percentage points
• Three flexible deployment options on Databricks let you balance control, scalability, and ease of use based on your specific requirements
• Vision Language Models (VLMs) can process complex medical documents, including handwritten notes and scanned images, with accuracy preferred by medical practitioners 46–175% more often than GPT-4o
• All deployments are HIPAA/GDPR compliant with no data leaving your secure environment — a critical advantage over cloud-based APIs
• Model sizes range from 2B to 70B parameters, allowing you to choose the right balance of accuracy and speed for your GPU resources and budget
• Specialized medical models (NER, RAG, SOAP generation) provide domain-specific capabilities not available in general-purpose LLMs
• Production-ready performance with 80–300 tokens/sec on modern GPUs and auto-scaling capabilities for variable workloads

Complete Notebooks:

• Loading Medical LLMs (Text-Only Models)
• Multi-Modal Vision Language Models
• Medical LLM on Databricks

📖 Documentation:

• Databricks Model Serving Overview
• John Snow Labs Medical LLM Documentation