notebooks/search_data_workflow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/rcsb/py-rcsb-api/blob/master/notebooks/search_data_workflow.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RCSB PDB Data API: Search and Data API Workflow Demo\n",
    "\n",
    "This quick-start notebook will walk through the basics of making queries in this package using a simple example. For more in-depth documentation reference the [README](https://github.com/rcsb/py-rcsb-api/blob/master/README.md).\n",
    "\n",
    "\\\n",
    "install the package: \n",
    "\n",
    "```pip install rcsb-api```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install rcsb-api"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this demo, we are interested in finding potential drugs to treat COVID-19 and the associated literature in order to conduct further research. To do this, we will:\n",
    " 1. Construct a query to fetch COVID-19 viruses with ligands bound (Python Search API package)\n",
    " 2. Find information about each ligand (PDB ID, associated publication titles, links to publications) (Python Data API package)\n",
    " 3. Parse our results and output in an easy-to-read format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python Search API: Find COVID-19 Structures with Ligand Bound\n",
    "To learn more about using the Search API Python package, read the documentation [here](https://rcsbsearchapi.readthedocs.io/en/latest/).\n",
    "\n",
    "We'll start by constructing a query.\n",
    "- Source organism is \"COVID-19 virus\" \n",
    "- Nonpolymer_enitity that is the subject of investigation in the structure\n",
    "- Modified chemical component is present"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['6W61', '7ARF', '7AWU', '7C8B', '7JP0', '7JPZ', '7JQ0', '7JQ1', '7JQ2', '7JQ3']\n"
     ]
    }
   ],
   "source": [
    "from rcsbapi.search import search_attributes as attrs\n",
    "\n",
    "# Create each subquery\n",
    "q1 = attrs.rcsb_entity_source_organism.taxonomy_lineage.name == \"COVID-19 virus\"\n",
    "q2 = attrs.rcsb_nonpolymer_entity_annotation.type == \"SUBJECT_OF_INVESTIGATION\"\n",
    "q3 = attrs.rcsb_polymer_entity_feature_summary.type == \"modified_monomer\"\n",
    "\n",
    "# Combine using bitwise operators (&, |, ~, etc)\n",
    "query = q1 & q2 & q3\n",
    "\n",
    "# Call the query to execute it\n",
    "result_list = query()\n",
    "\n",
    "# Save and print the first ten resilts\n",
    "short_result_list = (list(result_list)[0:10])\n",
    "print(short_result_list)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python Data API: Find Information About Structures\n",
    "\n",
    "Once we have the PDB IDs, we can query them using the Data API for information related to the structure. \n",
    "\n",
    "In this case, we will find the following for the first 10 results:\n",
    "- ID\n",
    "- Chemical component IDs\n",
    "- Whether the chemical component is the subject of investigation\n",
    "- Title of associated publication\n",
    "- Digital Object Identifier (DOI) if applicable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-10-22 15:43:39,181 [WARNING]-data_schema._construct_query_rustworkx: \n",
      "Some paths are being autocompleted based on the current API. If this code is meant for long-term use, use the set of fully qualified paths below:\n",
      "    [\n",
      "        \"rcsb_id\",\n",
      "        \"nonpolymer_entities.nonpolymer_entity_instances.rcsb_nonpolymer_entity_instance_container_identifiers.comp_id\",\n",
      "        \"nonpolymer_entities.nonpolymer_entity_instances.rcsb_nonpolymer_instance_validation_score.is_subject_of_investigation\",\n",
      "        \"citation.title\",\n",
      "        \"citation.pdbx_database_id_DOI\",\n",
      "    ]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'data': {'entries': [{'rcsb_id': '6W61',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'CL'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]}]},\n",
       "     {'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'EDO'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]},\n",
       "       {'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'EDO'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]},\n",
       "       {'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'EDO'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]}]},\n",
       "     {'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'SAM'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]},\n",
       "     {'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'ZN'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]},\n",
       "       {'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'ZN'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]}]}],\n",
       "    'citation': [{'title': 'The crystal structure of nsp10-nsp16 heterodimer from SARS-CoV-2 in complex with S-adenosylmethionine',\n",
       "      'pdbx_database_id_DOI': '10.1101/2020.04.17.047498'}]},\n",
       "   {'rcsb_id': '7ARF',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'RVW'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]},\n",
       "     {'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'DMS'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]},\n",
       "       {'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'DMS'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]}]},\n",
       "     {'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'CL'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]}]}],\n",
       "    'citation': [{'title': 'X-ray screening identifies active site and allosteric inhibitors of SARS-CoV-2 main protease.',\n",
       "      'pdbx_database_id_DOI': '10.1126/science.abf7945'},\n",
       "     {'title': 'Inhibition of SARS-CoV-2 main protease by allosteric drug-binding',\n",
       "      'pdbx_database_id_DOI': '10.1101/2020.11.12.378422'}]},\n",
       "   {'rcsb_id': '7AWU',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'S8B'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]},\n",
       "     {'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'CL'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]}]}],\n",
       "    'citation': [{'title': 'X-ray screening identifies active site and allosteric inhibitors of SARS-CoV-2 main protease.',\n",
       "      'pdbx_database_id_DOI': '10.1126/science.abf7945'},\n",
       "     {'title': 'Inhibition of SARS-CoV-2 main protease by allosteric drug-binding',\n",
       "      'pdbx_database_id_DOI': '10.1101/2020.11.12.378422'}]},\n",
       "   {'rcsb_id': '7C8B',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'CL'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]}],\n",
       "    'citation': [{'title': 'Crystal structure of the SARS-CoV-2 main protease in complex with Z-VAD(OMe)-FMK',\n",
       "      'pdbx_database_id_DOI': None}]},\n",
       "   {'rcsb_id': '7JP0',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'DMS'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'N'}]}]},\n",
       "     {'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'VJA'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]}],\n",
       "    'citation': [{'title': 'Crystal structure of Mpro with inhibitor r1',\n",
       "      'pdbx_database_id_DOI': None}]},\n",
       "   {'rcsb_id': '7JPZ',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'GHX'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]}],\n",
       "    'citation': [{'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main Protease Inhibitors*.',\n",
       "      'pdbx_database_id_DOI': '10.1002/cmdc.202000924'}]},\n",
       "   {'rcsb_id': '7JQ0',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'VHV'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]}],\n",
       "    'citation': [{'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main Protease Inhibitors*.',\n",
       "      'pdbx_database_id_DOI': '10.1002/cmdc.202000924'}]},\n",
       "   {'rcsb_id': '7JQ1',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'VHJ'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]}],\n",
       "    'citation': [{'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main Protease Inhibitors*.',\n",
       "      'pdbx_database_id_DOI': '10.1002/cmdc.202000924'}]},\n",
       "   {'rcsb_id': '7JQ2',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'VHM'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]}],\n",
       "    'citation': [{'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main Protease Inhibitors*.',\n",
       "      'pdbx_database_id_DOI': '10.1002/cmdc.202000924'}]},\n",
       "   {'rcsb_id': '7JQ3',\n",
       "    'nonpolymer_entities': [{'nonpolymer_entity_instances': [{'rcsb_nonpolymer_entity_instance_container_identifiers': {'comp_id': 'VHP'},\n",
       "        'rcsb_nonpolymer_instance_validation_score': [{'is_subject_of_investigation': 'Y'}]}]}],\n",
       "    'citation': [{'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main Protease Inhibitors*.',\n",
       "      'pdbx_database_id_DOI': '10.1002/cmdc.202000924'}]}]}}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from rcsbapi.data import DataQuery as Query\n",
    "\n",
    "query = Query(\n",
    "    input_type=\"entries\",\n",
    "    input_ids=short_result_list,\n",
    "    return_data_list=[\n",
    "        \"entries.rcsb_id\",\n",
    "        \"rcsb_nonpolymer_entity_instance_container_identifiers.comp_id\",\n",
    "        \"is_subject_of_investigation\",\n",
    "        \"citation.title\",\n",
    "        \"citation.pdbx_database_id_DOI\"\n",
    "    ]  \n",
    ")\n",
    "query.exec()\n",
    "query.get_response()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parsing the Result\n",
    "\n",
    "The result of the request is returned in JSON format. We can refer to the JSON output to understand the data structure and then parse it for the information that is useful to us.\n",
    "In this case, we will\n",
    "- Confirm the subject of investigation and find the ID if it exists (comp_id)\n",
    "- Find the publication title \n",
    "- Construct a link to the publication using the DOI\n",
    "- Put these data into a dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'6W61': {'link': 'https://doi.org/10.1101/2020.04.17.047498',\n",
      "          'subject_of_investigation': 'SAM',\n",
      "          'title': 'The crystal structure of nsp10-nsp16 heterodimer from '\n",
      "                   'SARS-CoV-2 in complex with S-adenosylmethionine'},\n",
      " '7ARF': {'link': 'https://doi.org/10.1126/science.abf7945',\n",
      "          'subject_of_investigation': 'RVW',\n",
      "          'title': 'X-ray screening identifies active site and allosteric '\n",
      "                   'inhibitors of SARS-CoV-2 main protease.'},\n",
      " '7AWU': {'link': 'https://doi.org/10.1126/science.abf7945',\n",
      "          'subject_of_investigation': 'S8B',\n",
      "          'title': 'X-ray screening identifies active site and allosteric '\n",
      "                   'inhibitors of SARS-CoV-2 main protease.'},\n",
      " '7C8B': {'link': '',\n",
      "          'subject_of_investigation': 'CL',\n",
      "          'title': 'Crystal structure of the SARS-CoV-2 main protease in '\n",
      "                   'complex with Z-VAD(OMe)-FMK'},\n",
      " '7JP0': {'link': '',\n",
      "          'subject_of_investigation': 'VJA',\n",
      "          'title': 'Crystal structure of Mpro with inhibitor r1'},\n",
      " '7JPZ': {'link': 'https://doi.org/10.1002/cmdc.202000924',\n",
      "          'subject_of_investigation': 'GHX',\n",
      "          'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main '\n",
      "                   'Protease Inhibitors*.'},\n",
      " '7JQ0': {'link': 'https://doi.org/10.1002/cmdc.202000924',\n",
      "          'subject_of_investigation': 'VHV',\n",
      "          'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main '\n",
      "                   'Protease Inhibitors*.'},\n",
      " '7JQ1': {'link': 'https://doi.org/10.1002/cmdc.202000924',\n",
      "          'subject_of_investigation': 'VHJ',\n",
      "          'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main '\n",
      "                   'Protease Inhibitors*.'},\n",
      " '7JQ2': {'link': 'https://doi.org/10.1002/cmdc.202000924',\n",
      "          'subject_of_investigation': 'VHM',\n",
      "          'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main '\n",
      "                   'Protease Inhibitors*.'},\n",
      " '7JQ3': {'link': 'https://doi.org/10.1002/cmdc.202000924',\n",
      "          'subject_of_investigation': 'VHP',\n",
      "          'title': 'A Quick Route to Multiple Highly Potent SARS-CoV-2 Main '\n",
      "                   'Protease Inhibitors*.'}}\n"
     ]
    }
   ],
   "source": [
    "from rcsbapi.data import DataQuery as Query\n",
    "from pprint import pprint  # for easier-to-read output\n",
    "\n",
    "json = query.get_response()[\"data\"][\"entries\"]\n",
    "output_dict = {}\n",
    "\n",
    "# iterate through the result of each entry requested\n",
    "for entry_dict in json:\n",
    "    rcsb_id = entry_dict[\"rcsb_id\"]\n",
    "\n",
    "    # Check for non-polymer subject of investigation, then append to chem_id_list\n",
    "    for entity_dict in entry_dict[\"nonpolymer_entities\"]:\n",
    "        for instance_dict in entity_dict[\"nonpolymer_entity_instances\"]:\n",
    "            is_subject = instance_dict[\"rcsb_nonpolymer_instance_validation_score\"][0][\"is_subject_of_investigation\"]\n",
    "            if is_subject == \"Y\":\n",
    "                comp_id = instance_dict[\"rcsb_nonpolymer_entity_instance_container_identifiers\"][\"comp_id\"]\n",
    "\n",
    "    # Find publication title\n",
    "    title = entry_dict[\"citation\"][0][\"title\"]\n",
    "\n",
    "    # Construct link from DOI (only exists if paper has been published or is on preprint server)\n",
    "    base_link = \"https://doi.org/\"\n",
    "    doi_link = \"\"\n",
    "    if entry_dict[\"citation\"][0][\"pdbx_database_id_DOI\"] is not None:\n",
    "        doi_link += base_link + entry_dict[\"citation\"][0][\"pdbx_database_id_DOI\"]\n",
    "\n",
    "    # Add to dictionary\n",
    "    output_dict[rcsb_id] = {\"title\": title, \"link\": doi_link, \"subject_of_investigation\": comp_id, }\n",
    "\n",
    "pprint(output_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Try it for yourself\n",
    "Combining use of our Search and Data API packages can make programmatic access to RCSB PDB easier than ever!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}