Agenda

Round-table update on analysis for the Predictive processing data release manuscript

Please submit 2 slides maximum prior to the meeting at the discussion link at the bottom of this meeting page. Any analysis is welcome, including from new participants.

At the onset of the meeting, we will review the number of presentations to go over as well as introduce any new comer to the meeting.

Meeting Recording

.

Meeting Notes

Manuscript Contribution Workflow: Jerome showed Stefan where to contribute the statistical-methods text in the P3 OpenScope data-release manuscript and explained that contributors can edit the Markdown file directly or use a collaborative editing link that Jerome plans to provide.

Repository Location: The manuscript is maintained in the P3 GitHub repository as a Markdown-based document containing the current text and figures. Jerome directed Stefan to the relevant manuscript file, index.md, for adding the discussion of statistical rigor and the order of selection versus statistical testing.

Editing Process: Jerome explained that index.md can be edited directly because it is a plain-text Markdown file. Jerome also plans to send a collaborative editing link within roughly the next week so contributors can edit the manuscript more conveniently.

Mesoscope SNR Metric Evaluation: Davis presented comparisons of three candidate mesoscope ROI signal-to-noise metrics, while Jerome and Farzaneh Najafi requested revisions to the figure structure, examples, anatomical interpretation, and validation strategy.

Candidate Metrics: Davis compared three metrics calculated on the same mesoscope ROIs and sessions: the fraction of detected events exceeding four baseline-noise standard deviations, the 95th-percentile event amplitude divided by successive-difference noise, and the dF/F 95th-minus-50th percentile range divided by fast-timescale MAD noise. Davis also discussed an event-amplitude approach based on successive differences and clarified that the Kaiman method was not included in the current comparison.

Metric Associations: The discussion established that the ratio-based metrics correlate with event rate and ROI area, whereas the event-fraction metric is more strongly associated with median event amplitude and is less coupled to activity rate. Davis noted that rate correlations may partly reflect event-detection thresholds, while the fraction metric normalizes by the total number of detected events. The group also discussed depth trends, with Jerome cautioning that depth-related differences may reflect different neuron populations rather than ROI quality.

Figure Revisions: Farzaneh recommended reorganizing the figure to begin with how each metric is constructed, followed by an example session, low- and high-scoring ROI examples, and characterization across sessions and imaging planes. Farzaneh also proposed showing average event-triggered calcium transients for the upper and lower SNR percentiles, while Jerome suggested including examples representing sparse and high-firing cell populations and considering anatomical context such as blood vessels.

Interactive Exploration: Jerome proposed a dynamic viewer in which users can select one or more metrics, inspect the distribution of all units, and click individual points to display the corresponding trace. Davis agreed to work on the interactive component after completing the metric comparisons, with the goal of helping the group build intuition about what high- and low-scoring ROIs represent.

Manual Validation: Farzaneh explained that correlations alone do not establish ground truth and suggested manually evaluating ROIs in the upper and lower portions of each metric ranking. Davis considered whether a downstream decoding task could help break ties between metrics, but Stefan advised against selecting QC units based on the desired downstream outcome; downstream analyses may be used for comparison, not as the selection criterion.

QC Standard And Paper Scope: Stefan, Jerome, Farzaneh Najafi, and Alexander Maier debated whether the data-release paper should merely expose the available data or also recommend a practical QC standard, ultimately favoring a documented recommendation alongside transparent alternatives and limitations.

Recommended Standard: Stefan argued that the group should identify the best currently supported metric and threshold, acknowledge its imperfections, and provide it as a practical default for researchers who do not want to develop their own criteria. Alexander supported this approach, comparing it with how Allen Institute spike-sorting standards help researchers use a common endorsed dataset even though standards evolve.

Transparent Alternatives: Farzaneh noted that the paper could include both a visualization-oriented presentation of all available data and a recommended set of units, but emphasized that choosing a reliable metric requires additional work and that the current three candidates had not yet been demonstrated to be clearly superior. No final metric or threshold was selected during the meeting.

Educational Context: Jerome said the paper should explain how the recommendation was reached and describe the trade-offs involved in analyzing calcium-imaging measurements near the reliability threshold. The paper should clarify that calcium signals are not direct proxies for spikes and that detected spikes or sorted units are not perfectly reliable, so users understand the limitations rather than treating the recommended output as definitive.

Existing QC Model: Farzaneh described an upcoming preprint involving an interface and deep-learning QC model trained on multiple metrics and manually labeled ground-truth data from the lab. The model covers both excitatory and inhibitory datasets and uses both deconvolved and convolved two-photon data; Farzaneh indicated that the model might later support the data-release QC effort, although its timeline could extend beyond the current paper schedule.

Scope Decision: Jerome stated that the project is intended primarily to release well-documented data rather than establish the frontier of event-processing methods. Farzaneh agreed that an exhaustive comparison of every event-detection and QC method would exceed the paper's scope; the immediate plan is to make the existing analyses cohesive, choose a defensible processing/QC approach, document it, and iterate later.

Code Integration And Contribution Plan: Jerome asked Davis to package the SNR analyses for shared use in the P3 repository, with the team planning to combine the mesoscope, SLAP2, and other modality viewers into a common system for comparison and consensus-building.

Code Sharing: Davis agreed to share the SNR analysis code and create the interactive component. Jerome requested a pull request to the P3 data-release repository, ideally including one or two candidate static figures and, if feasible, a dynamic viewer based on derived data.

Contribution Guidelines: Jerome reviewed updated contribution instructions covering data, figures, and preliminary binary files. He said contributors should generally keep individual binary files below approximately 50–100 MB, use a fork to create pull requests for binary contributions, and allow Jerome to review file sizes so the repository does not become excessively large.

Common Viewer System: Jerome plans to collect the available SNR implementations for EFIS, SLAP2, and mesoscope data and aggregate them into a shared environment where participants can run and compare the metrics. This shared system is intended to support the group's eventual convergence on figures, recommendations, and documented criteria.

Near-Term Delivery: Davis said he would try to prepare the code or pull request during the current week, subject to other commitments. Jerome linked this work to the goal of producing a usable manuscript version and supporting figures on the project timeline.

Deconvolution And Event Detection Alignment: Marcel asked which deconvolution implementation and parameters should be used across SLAP2 and mesoscope analyses, prompting discussion of the current L0 pipeline, possible comparison with a Tolias/Microns method, and whether external collaborators should be consulted.

Current Pipeline: Jerome said the existing analyses use an L0 optimizer and that its parameters should be documented in the methods section. Marcel was directed to check the existing calculations and methods for the SLAP2 and mesoscope data.

Alternative Algorithm: Stefan reported that Tolias has used a different deconvolution algorithm because the Allen L0 approach was reportedly less stable for the Microns data. Stefan suggested comparing the two algorithms in parallel, while noting that differences in imaging rate could explain why an algorithm performs differently across datasets.

Collaboration Outreach: Jerome preferred inviting the external developers to show or explain their method rather than simply obtaining code or data. Stefan said he would need to identify the appropriate contact through Tolias or the Microns project, and Jerome offered to contact Peter Ruprecht regarding a recent deep-learning event-detection method validated against ground-truth data.

Event-Detection Scope: Jerome said a deep-learning method such as Peter Ruprecht's could be attractive if the project were pushing processing methods, but the data release should prioritize reliable documentation and characterization over being methodologically state of the art. Farzaneh agreed that a detailed comparison of many event-detection methods would likely constitute a separate project.

Project Schedule And Upcoming Presentations: Jerome set near-term dissemination and manuscript targets, including an SFN data-release poster and a first manuscript version around October 1, while the team planned receptive-field-analysis presentations for the following week.

SFN Poster: Jerome plans to submit a last-minute SFN poster about the data release during the current week. The poster is expected to focus on introducing the dataset rather than presenting extensive new analysis, and Jerome expects to use the current contributor list.

October Draft: Jerome requested a first manuscript version by October 1 because the related review revision is scheduled for resubmission then; having a data-release manuscript or preprint available at that point would support the review. Farzaneh said the date is feasible for a well-documented data release with visualizations, but not if the team undertakes a comprehensive new event-detection and QC program.

Next Meeting: The next meeting is planned to focus mainly on receptive-field analysis. Marcel said the team would organize internally after a recent workshop, and Nicholas Rodriguez offered to present slides; Jerome confirmed that this topic would be scheduled for the following week.

💬 Join the discussion for this page on GitHub