Navigating the UNM HSC Research Data Environment
I spent about four years trying to get clean patient-level data out of the UNM Health Sciences Center research infrastructure before I finally stopped fighting it and learned how to make it work. The library itself is genuinely useful, but the access path is not obvious and there are a handful of quirks that will waste your time if you run into them unprepared. The first thing you need is an active UNM affiliation. That means a NetID, IRB approval for your project, and a data use agreement that has actually been signed by someone with authority. I have seen people sit on the permissions side of things for six weeks waiting on signatures that nobody knew who to send to. If your project involves PHI, the IRB layer comes first. If it is de-identified data, the DUA alone might get you in the door faster. Once permissions are in place, log into the HSC library portal and request access through the research data management section. You will fill out a project description, list your institutional role, and pick the dataset you need. It sounds bureaucratic, but the system is not trying to make it hard. It is trying to prove you are authorized to touch what you are asking for.
When your request goes through, expect a two-week turn-around for standard datasets. The larger ones—anything pulling from UNM Hospital electronic health records or the state of New Medicaid claims—can take six to eight weeks because they require a separate data governance review. I found that drafting your request with a tight scope upfront saves you from coming back multiple times. Say exactly what variables you need, what timeframe, and what method you plan to use. Vague requests get sent back.
How the Platform Actually Works
The system is built around a managed query environment. You do not download raw data to your laptop unless the dataset is explicitly flagged as fully de-identified. Instead, you run queries inside their secure workspace, and outputs are reviewed before anything leaves the system. This is standard for any institution handling HIPAA data, and it is one of the reasons the process feels slow if you are used to just taking a CSV home and working in R. The interface includes a query builder for SQL, and it supports R and Python scripts uploaded through their web client. Most people end up using a hybrid approach. They write small queries in the builder to pull subsets, then spin up an R session for the analysis because the script environment is more flexible for things like tidying and modeling. One thing the portal does not advertise clearly is the timeout limit. Sessions drop after roughly ninety minutes of inactivity, and any unsaved work disappears. I once lost a week of cleaning code because I left it open while stepping away for coffee. I keep a terminal open with my most recent saved version and use version control religiously from day one. A simple git repo for your scripts prevents that kind of loss entirely.
Get the Full Details
Specific Problem I Ran Into
About a year into using the platform, I needed to link three datasets: outpatient visit records, lab results, and pharmacy fills. The join key they provide across these tables is a de-identified subject ID, not a patient MRN. That sounds fine until you realize the ID schema changes between datasets depending on when the data was extracted. One table uses a newer hash, another uses the older one, and the mapping file they host on the portal was missing two weeks of overlap. I spent three days going back and forth with the data team trying to figure out why my join counts were off. In the end, the workaround was straightforward but not documented anywhere obvious. I pulled a small sample from each table, exported the subject IDs as a crosswalk file, and asked the data governance office to regenerate the mapping. It took ten days. After that, my joins worked and the output matched what I expected. Going forward, I always check the mapping file version date before I write any join logic. If the extraction date and the mapping date do not line up, I flag it immediately instead of running a full query and discovering the error later.
Practical Tips Based on Experience
Write your analysis scripts outside the portal and test them on a publicly available dataset first. The UNM HSC portal hosts a few synthetic training datasets meant exactly for this purpose. Using them lets you catch syntax errors and logic mistakes before you waste a real query allocation. Query allocations are limited per semester. I usually batch my work into two or three large queries instead of ten small ones. Each query costs time and credits, and the review process slows things down when you submit frequently. A single well-written query that pulls exactly what you need is faster than a scattered approach. If you need to export tables, request exports in batches no larger than five thousand rows per file. Larger exports trigger an extra compliance check and add five to seven business days to your timeline. For bigger pulls, break it into chunks by date or by service type.
The portal also has a built-in statistics summary tool that is surprisingly capable. Before you build a custom model, run the dataset through it. It gives you frequency distributions, missingness patterns, and basic cross-tabs in about five minutes. It will save you from writing a script just to understand what you are working with.
What the System Does Not Handle Well
The platform struggles with longitudinal data that spans multiple departments. Data from the oncology clinic, the emergency department, and the primary care network each have different extraction schedules and variable naming conventions. Merging them requires manual reconciliation of field definitions. The data team can help, but response time averages three to five business days per question, so plan your analysis timeline around that bottleneck. Another limitation is that the portal does not support real-time data. Everything is a snapshot from the last scheduled extraction, usually monthly. If you are doing time-sensitive work like tracking an outbreak or measuring the impact of a policy change that happened last week, the data will lag. You need to know the extraction date of whatever dataset you are using and factor the delay into your conclusions. The search function for datasets is also weak. It relies on keyword matching against field names rather than semantic search. If you are looking for something related to pediatric asthma medication adherence, typing those terms might not surface the right table. You have to know the internal naming conventions, which means browsing the dataset catalog alphabetically or asking someone who has already done the work.
Common Mistakes People Make
I see a lot of researchers request access to the full dataset when a subset would do. The full request takes longer to approve and slows down your analysis. Start with the smallest scope that answers your question, then expand if you find you need more. Another mistake is not documenting your query logic. The review process requires you to submit a methods note explaining what each query does. If you submit raw code without context, it gets returned for clarification. Write a short paragraph describing your intent for every query you run. It makes the review faster and keeps your own notes organized. People also forget to check the data dictionary before they start analyzing. The variable labels in the portal sometimes differ from the labels in the accompanying documentation. I found myself coding a variable as binary when the documentation said it was ordinal. Fixing that after a full analysis is painful.
Alternatives If the Portal Slows You Down
If you are doing work that requires speed over compliance nuance, the Clinical Data Warehouse at UNM Hospital is another option. It has a different governance structure and sometimes quicker turn-around for approved researchers. The data fields are less granular than the research library, but for broad administrative studies it can be more efficient. For work that does not involve patient data, the New Mexico Behavioral Risk Factor Surveillance System data is freely available and covers a lot of the same population health questions. It is less detailed but it is yours the moment you download it. The HSC library system is not perfect, but it is the best structured option for accessing UNM-affiliated clinical data if you understand how to work within its constraints. Plan ahead, document everything, and keep your requests scoped. That approach cuts my usual project setup time down from about three weeks to roughly one week and a half.
