Yes—but the defensible novelty should be the clinical task and evaluation protocol, not simply “use evidential deep learning for OOD rejection.” Existing work already covers DR OOD detection with Dirichlet Prior Networks, near-OOD detection of other eye diseases with deep-kernel learning, retinal open-set rejection, and evidential ordinal DR grading (Nandy, 2021) (Siebert, 2023) (Wang, 2023) (El Bellaj, 2026). So I would not claim that the model family or general idea is unprecedented.

Stronger candidate novelty

Proposed title

Comorbidity-Aware Open-Set DR Grading: Evaluating Evidential Referral for Unseen Ocular Disease Without Missing Coexisting Diabetic Retinopathy

Reframed gap

Existing studies establish that uncertainty can help identify unfamiliar retinal inputs, and that other ocular diseases can be near-OOD for DR grading models (Wang, 2023) (Siebert, 2023). But your most useful distinction is not simply “DR versus not DR.” It is whether a system can abstain on an unfamiliar ocular condition while still handling images in which DR coexists with that condition. A blanket OOD rejection could send away the very cases where a DR grade is clinically needed.

A defensible candidate gap is therefore:

It remains to be established whether an evidential DR grader trained only on DR-grade labels can support clinically safe selective referral under disease-held-out and site-held-out testing—particularly whether it rejects previously unseen ocular disease without disproportionately rejecting images with coexisting DR.

This is a candidate, not a verified “first” claim: the abstracts reviewed do not settle whether a full-text study has already tested this exact co-occurrence-aware evaluation.

Research question

When trained on DR-grading images only, does an evidential ordinal classifier improve disease-held-out OOD referral over strong uncertainty baselines, while preserving grading performance and avoiding excessive rejection of DR-positive images with coexisting ocular disease?

What would make the work meaningfully novel

1. Test compositional cases, not only clean disease categories. Include at least three test groups: DR-only; non-DR ocular disease without DR; and DR plus a coexisting ocular disease. Grade DR and label the coexisting condition independently. This tests whether rejection is clinically selective rather than merely sensitive to any unusual-looking retina.
2. Hold out disease types and sites. Keep some non-DR disease categories out of threshold development and reserve them for the final test; also test images from an external site or camera. Report results per disease and site. This makes the test about generalization to unknowns, not recognition of diseases already used to tune the rejection threshold.
3. Use a clinically constrained operating point. Compare models at the same DR sensitivity or referral/coverage target, then measure OOD detection and false rejection of DR-positive comorbidity cases. Report selective risk versus coverage as well as per-disease sensitivity, OOD AUROC/AUPR and calibration. The core result should be whether evidential uncertainty adds safety at an acceptable referral burden—not just whether it improves one OOD metric.
4. Keep the comparison fair. Compare the evidential model with a softmax baseline and at least one stronger uncertainty method. Deep-kernel learning has already been evaluated for near-OOD samples containing other eye diseases in DR grading (Siebert, 2023). A retinal open-set model has also been evaluated on unseen retinal-disease categories (Wang, 2023). Include these as direct comparators only if the data and task definitions can be aligned.

Important design clarification: “Trained only on DR images” can mean that only DR-grade-labelled images update the classifier’s weights. You will still need a separate development/calibration set containing representative non-DR cases to select a rejection threshold; keep disease categories in the final unseen-disease test out of that calibration set. If no non-DR examples can be used even for calibration, you can still study threshold-free OOD ranking, but you cannot claim that a clinically useful rejection threshold has been validated for unknown diseases.

Go/no-go and novelty wording

This is a stronger project if you can obtain reliably labelled DR-only, non-DR-only, and coexisting-disease images, plus an external source for testing. If you cannot get the co-occurrence group, narrow the claim to disease-held-out OOD evaluation and do not claim you have shown the model is safe when diseases coexist.

Use cautious wording such as “We evaluate whether…” or “We propose a disease-held-out, comorbidity-aware evaluation…” rather than “the first model to…” until a full-text search confirms priority. The abstracts already show substantial overlap in OOD and uncertainty methods; what may distinguish your study is the co-occurrence-aware safety question, frozen-threshold disease-held-out test, and comparison at a fixed clinical referral constraint. The current evidence is abstract-based, so the precise novelty still needs a full-text check.

End-to-end plan: evidential DR grading with OOD rejection

Below is a concrete workflow for your stated project: train a DR grader on DR-grading images, and have it abstain/refer rather than force a DR grade when the fundus image is from another ocular disease. I assume you have a GPU and can use public datasets; since I do not know your degree level, compute budget, or access to clinical data, I give a feasible baseline-first plan and identify where a research team or larger dataset is needed.

First, lock the task definition

Write this in one sentence before coding: “The model predicts DR severity for gradable fundus images from the target DR population; when evidence indicates an input outside that learned DR-grade distribution, it returns ‘uncertain—refer for human review’ rather than a DR grade.”

Resolve two details now:

• Define grades. “0–5” inclusive means six labels. Many commonly used DR datasets describe five severity categories; do not silently map a five-grade dataset onto six labels. State the clinical scale, the definition for each grade, and whether “no DR” is included. Recent work already combines an evidential Dirichlet head with ordinal DR grading, so do not frame that combination alone as novel; position your contribution around the OOD design and evaluation (El Bellaj, 2026).
• Define what counts as OOD. At minimum, separate other ocular disease (e.g., glaucoma, cataract, hypertensive retinal changes), poor-quality/ungradable fundus images, and non-fundus images. These are different failure types. Your main endpoint should be other ocular diseases; report the others separately.

Dataset plan

ODIR data are paired by patient, and a study of the dataset describes cases with more than one ocular pathology (Li, 2021) (Noutaila, 2026). Preserve those multiple labels rather than forcing coexisting conditions into one mutually exclusive class. Split by patient, not image, so both eyes remain in the same partition.

Practical recommendation: build the pipeline first on APTOS (DR development) plus one OOD source; then add EyePACS/DDR and a second OOD source if access and compute permit. ODIR-5K and RFMiD contain multiple ocular conditions, but because each dataset has its own acquisition and population characteristics, an OOD-versus-ID difference could reflect dataset source rather than disease. Reduce that confound by evaluating DR and non-DR conditions within a shared dataset where valid labels permit, and by repeating on a second independent disease dataset. Treat an “hypertension” label as a candidate ontology that needs verification; do not assume it means hypertensive retinopathy.

Before downloading: verify current access, license/terms, allowed redistribution, exact label ontology, grade counts, camera/site metadata, patient IDs, and whether both eyes belong to a patient. I could not verify current dataset access terms because this search was restricted to academic databases and abstracts, not official dataset pages.

Organized project steps and deliverables

1. Freeze the protocol (1–2 pages). Specify target population, camera/image type, grade scale, what triggers rejection, target referral use, primary outcome, baselines, and split rules. Register or timestamp this protocol before looking at final test results.
2. Audit candidate datasets. Download a small sample; inspect the label files and image quality. Build a table of image count by patient, grade, disease, camera/site, and quality. Check duplicates across datasets with file hashes and perceptual image similarity; remove cross-split duplicates.
3. Create a dataset manifest. One row per eye image: anonymized patient ID, image path, dataset, site/camera if known, DR grade, other-disease labels, gradability, and split. Never use patient ID, dataset name, or folder as a model feature.
4. Lock patient-level splits. For the DR source, create train/validation/internal-test partitions by patient (e.g., 70/15/15 as a starting proposal, adjusted for available counts). Keep both eyes and any repeat images from one person in the same partition. Reserve entire OOD disease categories for final disease-held-out testing; do not use them for tuning.
5. Build a plain DR grader first. Train a standard softmax classifier and verify its grade metrics. If this fails, resolve label mapping, leakage, and class imbalance before adding evidential learning.
6. Add uncertainty/OOD methods. Keep the backbone, image pipeline, patient splits, and training budget as comparable as possible. Implement the evidential model and simple score-based baselines first; then add a stronger comparator if resources allow.
7. Choose thresholds only on development data. Use the ID validation data for model selection and a separate OOD development set for a rejection threshold if the protocol allows OOD calibration. Keep the disease categories in the final OOD test unseen during threshold selection. Freeze the threshold before final testing.
8. Run the locked tests. Evaluate internal ID, external DR, known-OOD development diseases (reported separately), and disease-held-out OOD final test. Add external-site tests and mixed DR-plus-other-disease cases if labels exist.
9. Quantify uncertainty in the results. Repeat training across multiple random seeds if feasible. Compute patient-level bootstrap confidence intervals; do not treat two eyes from the same patient as independent observations.
10. Audit errors and shortcuts. Review false accepts (OOD given a DR grade), false rejects (DR image rejected), grade-specific misses, and performance by camera/site/quality. Check whether uncertainty is tracking black borders, image compression, or dataset source instead of pathology.
11. Package a reproducible release. Preserve split manifests, configuration files, training logs, model checkpoints, threshold values, environment versions, and a script that regenerates every table and figure. Share only data and weights whose terms permit sharing.
12. Make a bounded conclusion. Claim OOD rejection for the diseases, image sources, and operating points you actually tested—not “all eye diseases” or “clinical safety” from a small public benchmark.

Methodology to implement

A. Preprocessing and data control

• Load RGB fundus images; apply a consistent resize/crop policy and normalization. Start with ImageNet normalization if using an ImageNet-pretrained backbone. Preserve original images and log each transform.
• Use mild augmentation only: small rotations, horizontal flips where laterality is not a target, modest brightness/contrast changes, and crop/scale variation. Avoid strong color transforms or blur that could erase small DR lesions.
• Do not assume CLAHE or other enhancement helps. If tested, treat it as a training-only ablation and apply the identical pipeline to validation/test images. Never use test labels to choose preprocessing.
• Keep an “ungradable” label or separate quality flag. A low-quality image should be referred for image-quality reasons, not mislabeled as an ocular disease OOD.

B. Model and comparator set

Use one manageable pretrained backbone (e.g., ResNet-50 or EfficientNet-B0) for the first experiment. Match the backbone and data splits across:

1. Softmax baseline: standard K-class DR grade classifier; score OOD by maximum softmax probability and predictive entropy.
2. Evidential model (main model): the head predicts evidence for each DR grade. For K grades, compute e = softplus(z), alpha = e + 1, expected class probabilities p_k = alpha_k / sum(alpha), and uncertainty u = K / sum(alpha). Higher u is the candidate reject signal. Train with a documented evidential Dirichlet loss and regularization; select one implementation and report it precisely. Evidential uncertainty is a hypothesis to evaluate, not a guarantee of OOD detection.
3. Additional baselines: energy score and, if feasible, a deep ensemble or deep-kernel-learning model. DKL matters because a published DR grading study specifically reports improved detection of near-OOD samples containing other eye diseases (Siebert, 2023). A retinal open-set system also reports uncertainty for unseen retinal categories (Wang, 2023); use it as a comparator only if its labels, training protocol, and implementation can be aligned.

Because DR grades are ordered, state whether the classifier is ordinary multiclass or ordinal. A practical primary version is a five-class model if using the common five-category scale; apply the same ordinal-aware auxiliary objective to model comparisons where technically appropriate. Do not interpret an ordinary class-probability head as automatically understanding that adjacent grades are less severe errors than distant grades.

C. Training and calibration protocol

• Train on DR-grading images only; the classifier’s learned weights must not use OOD disease labels if your stated design is “DR-only training.”
• Tune learning rate, epoch, regularization, and class weights using DR validation data only. Use early stopping based on a prespecified DR metric (e.g., macro-F1 or quadratic-weighted kappa), not OOD test performance.
• Calibrate OOD rejection separately from model training. If using an OOD development set to choose threshold τ, state that clearly; that set must not include the disease categories reserved for final unseen-disease testing. At inference: if u > τ, return “uncertain / refer”; otherwise return the highest-probability DR grade. Freeze τ before final test evaluation.
• If you refuse to use any OOD examples even for threshold calibration, you can test whether uncertainty ranks OOD cases above DR cases, but you cannot claim that a clinically useful rejection threshold was validated for unseen diseases.

D. Evaluation: report both grading and rejection

On DR in-distribution and external DR images: quadratic-weighted kappa, macro-F1, balanced accuracy, per-grade sensitivity/recall, confusion matrix, and calibration (e.g., Brier score or ECE). Report the fraction of DR images rejected at the chosen threshold.

On OOD images: AUROC and AUPR for OOD-versus-ID, FPR at a prespecified OOD sensitivity (e.g., 95%), and OOD sensitivity separately for glaucoma, cataract, hypertensive retinal disease, and each other label with adequate sample size. State the positive class and threshold.

For selective prediction: plot error/risk among accepted images against coverage (fraction receiving a DR grade). Report referral rate, accepted-case grading performance, false-accept rate for OOD, and false-reject rate for ID DR images. Show results at a fixed operating point selected on development data—not only the best test-set threshold. A DKL DR study has already investigated other-eye-disease near-OOD (Siebert, 2023), and a retinal open-set study has tested unseen disease categories (Wang, 2023), so evaluate against baselines rather than presenting OOD AUROC alone as the contribution.

Implementation sequence (engineering checklist)

1. Environment: Python + PyTorch, timm for standard backbones, pandas for manifests, scikit-learn/TorchMetrics for metrics, and a fixed random-seed/config system. Record package versions and GPU details.
2. Repository layout: data_manifest/, configs/, src/data.py, src/models.py, src/losses.py, src/train.py, src/calibrate.py, src/evaluate.py, runs/, checkpoints/, and reports/. Keep raw image data outside version control.
3. Data loader: return image, DR grade, patient ID, dataset/site, disease labels, and quality flag. Apply transforms only after the split manifest is fixed. Assert no patient or duplicate image appears across splits.
4. Baseline: train the softmax model and save predictions—not just weights—for every split. Save per-image grade probabilities, labels, patient IDs, source, and quality. These cached predictions make method comparisons and error review reproducible.
5. Evidential head: replace the K-logit head with K evidence outputs, compute Dirichlet alpha and uncertainty, and train with your selected evidential objective. Unit-test that evidence is nonnegative, probabilities sum to one, and u lies in (0,1].
6. Score functions: implement maximum softmax probability, entropy, evidential uncertainty, energy, and ensemble disagreement if included. Check score direction so larger consistently means “more OOD.”
7. Calibration script: read only allowed development predictions, select threshold under a stated criterion (for example, meet a minimum DR sensitivity while maximizing OOD sensitivity), save τ to a locked JSON config, and prevent evaluation code from changing it.
8. Evaluation script: generate metrics and plots separately by disease, dataset/site, grade, quality, and operating point. Bootstrap by patient. Keep a final test lock: no repeated threshold or hyperparameter changes after viewing its results.
9. Ablations: compare EDL vs softmax under identical backbone/splits; remove OOD calibration; test different OOD thresholds; compare within-source versus cross-source OOD; assess whether preprocessing and class imbalance handling change ID and OOD behavior.
10. Error review: create a blinded review sheet of false accepts and false rejects for an ophthalmologist or trained grader if available. Do not label model uncertainty as clinical uncertainty without human adjudication.

Suggested first milestone and decision gates

Milestone 1: complete a dataset/label audit and reproduce one five-grade DR baseline on APTOS. Gate: do not continue until patient-level splits, class mapping, and saved predictions are verified.

Milestone 2: test MSP/entropy OOD scoring on a disease-containing dataset and one separate source. Gate: if apparent OOD performance collapses when comparing within a shared source, dataset confounding is likely; redesign the split before developing the evidential method.

Milestone 3: implement EDL and compare against the exact same softmax backbone. Gate: continue the EDL-specific contribution only if it improves OOD rejection or accepted-case safety at comparable DR grading/coverage—not merely training confidence or overall accuracy.

Milestone 4: lock threshold, evaluate held-out diseases and external data, then write up limitations. Public image datasets are a research benchmark, not clinical validation; a deployment claim would require prospective, representative clinical evaluation.

The key feasibility risk is not the neural network code; it is reliable, correctly matched OOD labels and splits. RFMiD’s abstract reports multiple cameras and expert-adjudicated labels, while ODIR-5K contains paired images and multi-disease cases—both are useful but require patient-level splitting and careful label interpretation (Pachade, 2021) (Li, 2021). Also note that prior work already combines evidential ordinal DR grading across datasets and prior DKL work already targets other-eye-disease near-OOD (El Bellaj, 2026) (Siebert, 2023). Your study should therefore present a controlled evaluation of whether evidential uncertainty actually improves rejection for the specific diseases and datasets tested, rather than claiming the general combination is unprecedented.