Esophageal Carcinoma

Written and checked by a reviewer, not generated from metadata.

No reviewed research questions yet. The measurements and field completeness above are machine-extracted and still useful for judging fit.

Written by a reviewer against the source, not inferred from what is missing.

No reviewed limitations yet. Check the field completeness chart above before committing to an analysis. Low coverage on an endpoint you need is the most common blocker.

Articles that analyzed these data, counted separately from articles that only cited the paper. Only the first counts as reuse.

Analyses someone ran on these data end to end, each with a receipt recording when it ran, against which packages and for how long.

Nobody has shared an analysis of this dataset yet. The examples on other datasets cover the common patterns and can be adapted - and if you adapt one, it belongs here.

Starter code generated from this record’s own identifiers, then the access steps.

import json, requests

PROJECT = "TCGA-ESCA"
filters = {"op": "and", "content": [
    {"op": "in", "content": {"field": "cases.project.project_id", "value": [PROJECT]}},
    {"op": "in", "content": {"field": "access", "value": ["open"]}},
]}
params = {"filters": json.dumps(filters),
          "fields": "file_id,file_name,data_category,data_type,file_size",
          "size": "10000", "format": "JSON"}
files = requests.get("https://api.gdc.cancer.gov/files", params=params, timeout=60).json()["data"]["hits"]
print(len(files), "open files")

# Write a manifest and fetch with the GDC client (https://gdc.cancer.gov/access-data/gdc-data-transfer-tool)
with open("manifest.txt", "w") as fh:
    fh.write("id\n" + "\n".join(f["file_id"] for f in files))
# $ gdc-client download -m manifest.txt

import json, requests

filters = {"op": "in", "content": {"field": "project.project_id", "value": ["TCGA-ESCA"]}}
params = {"filters": json.dumps(filters), "size": "5000", "format": "TSV",
          "fields": ",".join([
              "submitter_id", "demographic.vital_status", "demographic.days_to_death",
              "diagnoses.days_to_last_follow_up", "diagnoses.ajcc_pathologic_stage",
              "diagnoses.treatments.treatment_type", "diagnoses.treatments.treatment_outcome"])}
tsv = requests.get("https://api.gdc.cancer.gov/cases", params=params, timeout=60).text
open("TCGA-ESCA_clinical.tsv", "w").write(tsv)

from idc_index import IDCClient

COLLECTION = "tcga_esca"
client = IDCClient()

series = client.index[client.index["collection_id"] == COLLECTION]   # one row per DICOM series
print(len(series), "series,", series["PatientID"].nunique(), "patients,",
      round(series["series_size_MB"].sum() / 1024, 1), "GB")
print(series["Modality"].value_counts().to_dict())

# Everything above is an index lookup and costs nothing. Uncomment to pull the pixels;
# the client refuses up front if the disk cannot hold them.
# client.download_from_selection(collection_id=COLLECTION, downloadDir="./tcga_esca")

curl -s https://cd2s.vercel.app/data/datasets/gdc-tcga-esca.json | python3 -c '
import json, sys
r = json.load(sys.stdin)
for l in r["limitations"]:
    print(l["severity"].upper(), "-", l["statement"][:120])
print("survival endpoint:", r["longitudinal"]["has_survival_endpoint"], "| treatment response:", r["longitudinal"]["has_treatment_response"])'

Access steps

1. 1

   Open the project in the GDC Data PortalFor people5 minutes

   Check the portal's current case and file counts before requesting anything; the cohort can change after this record was built.

   https://portal.gdc.cancer.gov/projects/TCGA-ESCA

   Generated from repository policy. Check the linked policy before downloading.

2. 2

   Download the open files without an accountFor peopleminutes to hours, by cohort size

   4,647 open-access files are marked open access. Add them to a portal cart, export a manifest, and fetch it with the GDC Data Transfer Tool.

   gdc-client download -m gdc_manifest.txt

   https://gdc.cancer.gov/access-data/gdc-data-transfer-tool

   Generated from repository policy. Check the linked policy before downloading.

3. 3

   Request controlled access through dbGaPFor peopledays to a few weeks after signing official approval

   6,523 controlled-access files require an approved data access request. The request starts at dbGaP, not in the portal. The relevant study is phs000178.

   Requires: eRA Commons account, institutional signing official approval, dbGaP data access request for phs000178

   https://gdc.cancer.gov/access-data/obtaining-access-controlled-data

   Generated from repository policy. Check the linked policy before downloading.

4. 4

   Download controlled files with your tokenFor people

   An approved request yields a token. The same manifest fetches nothing controlled without it.

   Requires: approved dbGaP request, GDC authentication token

   gdc-client download -m gdc_manifest.txt -t gdc-user-token.txt

   https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/study.cgi?study_id=phs000178

   Generated from repository policy. Check the linked policy before downloading.

5. 5

   Query the project from code, with no credentialsFor agents

   The file index is public even where files are not, so an agent can size the cohort before anyone requests access.

   curl -s 'https://api.gdc.cancer.gov/files' --get \
     --data-urlencode 'filters={"op":"in","content":{"field":"cases.project.project_id","value":["TCGA-ESCA"]}}' \
     --data-urlencode 'fields=file_id,file_name,data_type' \
     --data-urlencode 'size=10000' --data-urlencode 'format=TSV'

   https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started/

   Generated from repository policy. Check the linked policy before downloading.

6. 6

   Pass the token when the agent needs controlled filesFor agents

   Without a token, an API query may return only the open subset. Compare returned counts with the record before treating them as the whole cohort.

   Requires: GDC authentication token

   curl -H "X-Auth-Token: $GDC_TOKEN" https://api.gdc.cancer.gov/data/<file_id>

   https://docs.gdc.cancer.gov/API/Users_Guide/Getting_Started/

   Generated from repository policy. Check the linked policy before downloading.

All of it, and more, is in this record’s JSON.

Same disease area, favoring cohorts that add a measurement this one lacks.