Skip to content

Inclusion of an overhauled version of the first PyXspec walkthrough notebook #260

Open
DavidT3 wants to merge 56 commits into
mainfrom
notebook/ingestPyXspecNB1
Open

Inclusion of an overhauled version of the first PyXspec walkthrough notebook #260
DavidT3 wants to merge 56 commits into
mainfrom
notebook/ingestPyXspecNB1

Conversation

@DavidT3
Copy link
Copy Markdown
Collaborator

@DavidT3 DavidT3 commented Apr 8, 2026

The first step in making the PyXspec tutorials and demonstrations available through HEASARC-tutorials (and thus on Fornax as well), a revamped version of the first PyXspec walkthrough notebook.

The overall content currently remains the same, no major sections have been added or removed (though we might consider some changes on that front).

All plotting code has been altered.

Some previous bugs have been fixed (the contour plot had mislabeled axes, and wasn't showing the 1, 2, and 3 sigma levels that was claimed in the commentary).

More commentary has been added at certain points.

The notebook has been put into the markdown format required by HEASARC-tutorials, and now matches the template in style. Sections and subsections have been introduced.

Science review checklist

  • Does using high-energy data make up a significant part of the tutorial?
  • Is there a use case in the introduction that motivates the code?
  • Does the code do what the introduction says it is going to do?
  • Is it scientifically accurate?
  • Have all necessary references to literature been included?

Formatting checklist

  • Did you base your notebook on the HEASARC-tutorials template?
  • Are all sections in the HEASARC-tutorial template included in your notebook?
  • Is the notebook title compact and informative? It will be the name of the notebook on the HEASARC website!
  • Have you populated the notebook front-matter (the metadata at the top of the notebook)?
  • Is the kernel specified in the front-matter (e.g., heasoft, sas, ciao) correct for the notebook?
  • Have you added an entry for your notebook in the *_index.md file for the containing directory?
  • Is all plotting matplotlib (or alternative) plotting code in isolated cells, and do they have the metadata required to collapse the code when the notebook/website is viewed?

Tech review checklist

  • Documentation:
    • Is every function documented?
    • Do all code cells have corresponding narratives/comments?
    • Are all code comments of a purely technical nature? All narratives should be in Markdown cells.
    • Did you populate the 'Runtime' section?
  • Notebook execution, error handling, etc.:
    • Does the notebook run end-to-end, out of the box?
    • Are errors handled appropriately, with try/except statements that are narrow in scope?
    • Have warnings been dealt with appropriately, preferably by updating the code to avoid them (i.e., not by simply silencing them)?
  • Efficiency:
    • Is data accessed from the cloud where possible?
    • Is the code parallelized where possible?
    • If the notebook is intended to be scaled up, does it do that efficiently?
    • Is memory usage optimized where possible?
  • Cleanup:
    • Have blocks of code that need to be re-used been turned into functions and placed in the 'global setup'-'function' section?
    • Has unused code been removed (e.g., unused functions and commented-out lines)?
    • Are comment lines wrapped so all fit within a max of 90 - 100 characters per line?
    • Do plots use color-blind friendly palettes for plotting? Try this simulator for a visual check.

David Turner added 30 commits April 2, 2026 12:30
…o the HEASARC-tutorials resource. Content is essentially identical to the original in this first pass, and as such will not execute (data files are missing, etc.). Some changes to imports and line lengths have had to be made however, just to pass the precommit checks. For issue #251
…e required spectrum and response file. Uses the S3 https link, and a built-in Python library. For issue #251
…removed the use of jupyter cell magic to change directories for the spectrum loading (we use contextlib instead now). For issue #251
…used in a more 'correct' (for want of a better word) way now. For issue #251
…dness of fit test simulations. Rather than extracting plotting values from PyXspec Plot, we fetch the actual chi-squared distribution array and make our own histogram. For issue #251
…ms of making the legend nicer/clearer. For issue #251
…etic purposes. Couple of small typo fixes.
…otted as different lines, with a 'total' output being the combined model. Should ask if there is a dynamic way to retrieve only additive model components from PyXspec, for fully automatic plotting like this. For issue #251
…ow to calculate delta-chi-square values for specific confidence percentages (1, 2, and 3 sigma in this instance). For issue #251
…ontour level mistype and to show the user how to calculate delta-chi-square values for particular probabilities. Next big step is to go through and make plotting data safer by storing it in differently named variables. For issue #251
… commented out legacy plotting code. For issue #251
…some commentary and added more for context. Also stored the retrieved plotting data in a dictionary rather than in variables that are in danger of being re-used. For issue #251
…a stepped model line in the renormalized model plot. For issue #251
…on to plot spectrum, model, and residuals. That happens a few times in this notebook so we might as well avoid repeating code. For issue #251
…alization, makes more sense to me in that order.
…ative variable names even if we never user them.
…subsection names to the alternative model testing section - makes it much easier to navigate the different models.
…an just the theoretical models. Also tried collapsing the cell output of the steppar call, as it gets so long. For issue #251
… avoid steppar outputs appearing in the next cell's outputs when running all - I NEED TO TELL CRAIG ABOUT THAT. For issue #251
@DavidT3
Copy link
Copy Markdown
Collaborator Author

DavidT3 commented Apr 8, 2026

Happily it executed fine - time for polishing.

I've added the 'skip-doc-build' label to disable triggering automatic rebuilds on any commits I make until the badge is removed. The free resources we get from CircleCI are limited, so we try to be as efficient as possible.

@DavidT3
Copy link
Copy Markdown
Collaborator Author

DavidT3 commented Apr 8, 2026

**The nH problem - ** the notebook points to a column density value of $4x10^{22}$, but I'm not sure where that comes from. The nhtool (from all 3 currently available maps) result is around 1.25

This is an important part of the demonstration that leads into showing how/why you might construct and fit a two-additive-component model, but the reason may be a little artificial as it stands.

I would be alright with that, but actually in the new version of the separate-model-component plot I also show the data, and it is still a pretty poor fit at the low end. Possible some of this is driven by my changing to TBabs, but something needs to change about this I think.

image

I'll see how fixing it at the current nH result from nhtool changes things.

David Turner added 12 commits April 8, 2026 09:32
…otebook repo, changed the last updated date (really need to get around to issue #163), and fixed the path to the global data directory in 'configuration'. For issue #251
…xample dataset a hyperlink rather than just a URL.
… of it into bullet points, which I personally find easier to read. For issue #251 and PR #260
…loat values being passed to ignore/notice. Added more links back to previous figures we refer to. For issue #251 and PR #260
…een the end of the output text table, and the markdown commentary (it helps improve the clarity in the Jupyter notebooks, though it wasn't a problem in the rendered notebook). For issue #251 and PR #260
… in subsubsections now, which I hope will be a little easier to parse. Also shifted some of that commentary to the goodness of fit section, and also altered that section's title. Finally, added a link in the additional resources section that goes to the general page where the parallel settings are defined in a docstring. For issue #251 and PR #260
… spectrum, model, and residuals. For issue #251 and PR #260
…ion of XSPEC error strings, and a demonstration of how to dynamically retrieve the confidence range values from the model object. For issue #251 and PR #260.
… so PyCharm's grammar checker will stop passive-aggressively highlighting them. For PR #260
@DavidT3 DavidT3 added ready-for-review HEASARC internal review process can begin and removed skip-doc-build Do not trigger automatic CI/CD building of documentation for this PR. labels Apr 8, 2026
@DavidT3 DavidT3 changed the title [EARLY]: Inclusion of an overhauled version of the first PyXspec walkthrough notebook Inclusion of an overhauled version of the first PyXspec walkthrough notebook Apr 8, 2026
@DavidT3
Copy link
Copy Markdown
Collaborator Author

DavidT3 commented Apr 8, 2026

@xspector @krutkow @c181gordon

I've added you all as potential reviewers - the notebook as it stands is pretty complete, though I admit that I now have XSPEC/notebook fatigue and it might still need some more attention particularly towards the end.

Something I wanted to talk about is the multi-additive-component model demonstration in this notebook; the column density currently used for the fixed nH is quite a bit larger than the current HI4PI value from nH, which sort of undermines the point being made about how the absorbed powerlaw is under predicting soft counts (particularly because the two-additive-component fit doesn't look like much of a better fit).

Clearly the tutorial needs a demonstration of a two continuum component fit, but any thoughts on what we should do about that?

David Turner added 4 commits April 9, 2026 09:37
…max depth of the PyXspec index page toctree, to see if that brings back the subsections in the pyxspec-basics.md table of contents in the rendered version. For PR #260
@DavidT3 DavidT3 added the skip-doc-build Do not trigger automatic CI/CD building of documentation for this PR. label May 4, 2026
@DavidT3 DavidT3 removed the skip-doc-build Do not trigger automatic CI/CD building of documentation for this PR. label May 4, 2026
David Turner added 2 commits May 4, 2026 12:20
…-basics.md, while also fixing a bug that would have resulted in the wrong label being applied to a smooth model line. For PR #260
… function defined in pyxspec-basics.md. Also made sure that the non-staircase model line plotter had the same linewidth as the staircase plotter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-content Changes or additions to the content of the documentation multi-mission Issues that relate to multiple high-energy missions ready-for-review HEASARC internal review process can begin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ingest PyXspec notebook tutorial ONE

1 participant