Reporting Checklist For Nature Neuroscience: Statistics Reporting, by Figure

Stephen Smith
# Main Figures:
Manuscript Number:
NN-BC52986
# Supplementary Figures:
Manuscript Type:
Brief Communication
# Supplementary Tables:
# Supplementary Videos:
nature neuroscience | reporting checklist
Corresponding Author:
Reporting Checklist for Nature Neuroscience

This checklist is used to ensure good reporting standards and to improve the reproducibility of published results. For more information, please
read Reporting Life Sciences Research.
Please note that in the event of publication, it is mandatory that authors include all relevant methodological and statistical information in the
manuscript.
Statistics reporting, by figure

Please specify the following information for each panel reporting quantitative data, and where each item is reported (section, e.g. Results, &
paragraph number).
Each figure legend should ideally contain an exact sample size (n) for each experimental group/condition, where n is an exact number and not a
range, a clear definition of how n is defined (for example x cells from x slices from x animals from x litters, collected over x days), a description of
the statistical test used, the results of the tests, any descriptive statistics and clearly defined error bars if applicable.
z

z
For any experiments using custom statistics, please indicate the test used and stats obtained for each experiment.

z
Each figure legend should include a statement of how many times the experiment shown was replicated in the lab; the details of sample
collection should be sufficiently clear so that the replicability of the experiment is obvious to the reader.

z
For experiments reported in the text but not in the figures, please use the paragraph number instead of the figure number.
Note: Mean and standard deviation are not appropriate on small samples, and plotting independent data points is usually more informative.
When technical replicates are reported, error and significance measures reflect the experimental variability and not the variability of the biological
process; it is misleading not to state this clearly.
Fig.
legend
9, 9, 10,
15
results,
para 6
unpaired ttest
Results
para 6
15
mice from at least 3 Methods

litters/group
para 8
slices from 10 mice
Results
para 6
error bars are

mean +/- SEM
error bars are
mean +/- SEM
EXACT VALUE
Fig.
legend
p = 0.044
Results
para 6
p = 0.0006
VALUE
SECTION &
PARAGRAPH #
one-way
ANOVA
REPORTED?
SECTION &
PARAGRAPH #
SECTION &
PARAGRAPH #
example
1a
DEFINED?
SECTION &
PARAGRAPH #
FIGURE
NUMBER
EXACT
VALUE
Fig.
legend
F(3, 36) = 2.97
Fig. legend
Results
para 6
t(28) = 2.808
Results
para 6
April 2015
WHICH TEST?
(AVERAGE, VARIANCE)
DEGREES OF
FREEDOM &
F/t/z/R/ETC VALUE
P VALUE
SECTION &
PARAGRAPH #
DESCRIPTIVE STATS
example
TEST USED
1
Nature Neuroscience: doi:10.1038/nn.4125
CCA
(Canonical
Correlation
Analysis)
see
above
main
text
paras
4-5
see
above
see above
illustrative plots
based on the
Fig.
top-30
legen
connectome edge
d and
weights
Supp
see above
(equivalent to
Figs
thresholding full
3,4
list of edges at the
legend
99.85th
percentile)
see
above
see above
see above
see above
EXACT VALUE
VALUE
SECTION &
PARAGRAPH #
REPORTED?
DEGREES OF
FREEDOM &
F/t/z/R/ETC VALUE
P VALUE
SECTION &
PARAGRAPH #
DEFINED?
SECTION &
PARAGRAPH #
SECTION &
PARAGRAPH #
EXACT
VALUE
(AVERAGE, VARIANCE)
correlations and %
variance
explained, for
main text
subject measures
Fig.
paras 3-5;
461
vs CCA canonical
legend
Online
subjects;
variates. 1c also
All HCP-500
; main
Methods
158
shows CCA
text
subjects with
(through
subject
components' %
paras
complete rfMRI
out and
measures
variance explained
timeseries data,
4-5;
in
; 19900
of total original
with no exclusions
Online
particular
connecto
data matrices and
Metho
section
me edges
permutationds
"Data")
based null
distributions of
same.
CCA
connectome
edges
weights (all
Fig.
quantitative
legend
issues
addressed
above for fig
1)
all
All
quantitative
suppl
issues are
+ eme
same as
- ntal addressed
figur
above for fig
es
1
SECTION &
PARAGRAPH #
FIGURE
NUMBER
+ 1a,b,
- c
WHICH TEST?
DESCRIPTIVE STATS
main text
para 4;
For this CCA
157-461 (See
Online
mode:
Online Methods;
Methods
P=10^-5
DoF and multiple
sections
main
comparison
"CCA
(permutation text para
correction
modeling
test
4
directly
" and
accounting for
accounted for by
"Addition
family
permutation
al CCA
structure)
testing)
validation
tests"
see above
see
above
see above
see
above
see
above
see above
see
above
see above
see
above
TEST USED
Representative figures
1. Are any representative images shown (including Western blots and

immunohistochemistry/staining) in the paper?
Figs 1,2
If so, what figure(s)?
2. For each representative image, is there a clear statement of

how many times this experiment was successfully repeated and a
discussion of any limitations in repeatability?
If so, where is this reported (section, paragraph #)?
April 2015
All 461 complete subjects' datasets were used and the CCA is
deterministic (and hence only run once) - these whole-analysis
summary images are illustrative of the strongest connectome
nodes/edges. However, see Online Methods ("Additional CCA
validation tests") for details of several analysis variants also applied,
all giving equivalent results.
2
Statistics and general methods
1. Is there a justification of the sample size?
N/A (all datasets used; exploratory mutivariate analysis)
If so, how was it justified?

Where (section, paragraph #)?
Even if no sample size calculation was performed, authors should
report why the sample size is adequate to measure their effect size.
2. Are statistical tests justified as appropriate for every figure?

Primary CCA mode result is tested for significance using

permutation testing that accounts for family structure and multiple
comparisons (main text paras 4-5; Fig 1 legend; Online Methods
throughout).
a. If there is a section summarizing the statistical methods in

the methods, is the statistical test for each experiment
clearly defined?
yes (permutation test details, 100,000 permutations)
b. Do the data meet the assumptions of the specific statistical

test you chose (e.g. normality for a parametric test)?
yes - exchangeability (see Online Methods discussion of accounting

for family structure).
Where is this described (section, paragraph #)?
c. Is there any estimate of variance within each group of data? N/A

Is the variance similar between groups that are being
statistically compared?
d. Are tests specified as one- or two-sided?
correlation between CCA U and V columns: one-sided
e. Are there adjustments for multiple comparisons?
yes - full FWE correction for the 100 CCA modes estimated
3. Are criteria for excluding data points reported?

Was this criterion established prior to data collection?
4. Define the method of randomization used to assign subjects (or

samples) to the experimental groups and to collect and process data.
Yes - reported in detail in Online Methods section "CCA modeling".

Data used is a public resource; exclusions were carried out before
any of the main CCA analyses; see Online Methods section "CCA
modeling".
N/A
If no randomization was used, state so.

Where does this appear (section, paragraph #)?
N/A
April 2015
5. Is a statement of the extent to which investigator knew the group

allocation during the experiment and in assessing outcome included?
If no blinding was done, state so.
3
7. Is the species of the animals used reported?
N/A
8. Is the strain of the animals (including background strains of KO/

transgenic animals used) reported?
N/A
9. Is the sex of the animals/subjects used reported?
Yes - Online Methods section "Data".
6. For experiments in live vertebrates, is a statement of compliance with N/A

ethical guidelines/regulations included?
10. Is the age of the animals/subjects reported?
Yes - Online Methods section "Data".
11. For animals housed in a vivarium, is the light/dark cycle reported?
N/A
12. For animals housed in a vivarium, is the housing group (i.e. number of N/A
animals per cage) reported?
13. For behavioral experiments, is the time of day reported (e.g. light or
dark cycle)?
N/A
14. Is the previous history of the animals/subjects (e.g. prior drug

administration, surgery, behavioral testing) reported?
Fully described in the core HCP literature referenced here.
a. If multiple behavioral tests were conducted in the same

group of animals, is this reported?
N/A
15. If any animals/subjects were excluded from analysis, is this reported?
Yes - see above.
a. How were the criteria for exclusion defined?

April 2015
See above; complete details in Online Methods section "CCA

modeling".
4
Reagents
1. Have antibodies been validated for use in the system under study
(assay and species)?
a. Is antibody catalog number given?
N/A
N/A
b. Where were the validation data reported (citation,

supplementary information, Antibodypedia)?
b. Specify reasons for any discrepancy between the number of N/A

animals at the beginning and end of the study.
N/A
2. Cell line identity

a.
N/A
Are any cell lines used in this paper listed in the database of
commonly misidentified cell lines maintained by ICLAC and
NCBI Biosample?
b. If yes, include in the Methods section a scientific

justification of their use--indicate here in which section and
paragraph the justification can be found.
N/A
c. For each cell line, include in the Methods section a

statement that specifies:
N/A
- the source of the cell lines

- have the cell lines been authenticated? If so, by which
method?
- have the cell lines been tested for mycoplasma
contamination?
April 2015
5
Data deposition
Data deposition in a public repository is mandatory for:

a. Protein, DNA and RNA sequences
b. Macromolecular structures
c. Crystallographic data for small molecules
d. Microarray data
Deposition is strongly recommended for many other datasets for which structured public repositories exist; more details on our data policy are
available here. We encourage the provision of other source data in supplementary information or in unstructured repositories such as Figshare
and Dryad.
We encourage publication of Data Descriptors (see Scientific Data) to maximize data reuse.
1. Are accession codes for deposit dates provided?
HCP data used is already publicly released, as described in the main

text and Online Methods.
Computer code/software
Any custom algorithm/software that is central to the methods must be supplied by the authors in a usable and readable form for readers at the
time of publication. However, referees may ask for this information at any time during the review process.
1. Identify all custom software or scripts that were required to conduct

the study and where in the procedures each was used.
All software used (Matlab, FSL, permutation testing (Winkler

reference), FreeSurfer, nearestSPD) is identified in detail in Online
Methods. Also see below (2).
2. If computer code was used to generate results that are central to the
paper's conclusions, include a statement in the Methods section
under "Code availability" to indicate whether and how the code can
be accessed. Include version information as necessary and any
restrictions on availability.
All software used is publicly available, including the HCP

Connectome Analysis pipeline referenced (Marcus reference). The
custom matlab script will be made publicly freely available (see
Online Methods, end of CCA section).
Human subjects
1. Which IRB approved the protocol?

Where is this stated (section, paragraph #)?
2. Is demographic information on all subjects provided?
Fully described in the core HCP literature referenced here; the

paper is only using publicly available datasets.
Yes; see Online Methods (throughout).
3. Is the number of human subjects, their age and sex clearly defined?
Yes; see Online Methods (section "Data" and throughout).

April 2015
4. Are the inclusion and exclusion criteria (if any) clearly specified?
Yes (see above)
6
N/A
Where is this information described (section, paragraph #)?
6. Is a statement included confirming that informed consent was

obtained from all subjects?
Fully described in the core HCP literature referenced here; the

paper is only using publicly available datasets.
7. For publication of patient photos, is a statement included confirming

that consent to publish was obtained?
N/A
fMRI studies
5. How well were the groups matched?
For papers reporting functional imaging (fMRI) results please ensure that these minimal reporting guidelines are met and that all this
information is clearly provided in the methods:
1. Were any subjects scanned but then rejected for the analysis after the Yes - see above for full details regarding exclusions.
data was collected?
a. If yes, is the number rejected and reasons for rejection

described?
2. Is the number of blocks, trials or experimental units per session and/

or subjects specified?
See above and Online Methods throughout; 468 subjects were

initially considered and 7 were rejected because their timeseries
data were incomplete. Hence we did not reject any subjects where
there was a complete dataset.
N/A
3. Is the length of each trial and interval between trials specified?
N/A
4. Is a blocked, event-related, or mixed design being used? If applicable,

please specify the block length or how the event-related or mixed
design was optimized.
N/A (resting-state fMRI)
5. Is the task design clearly described?
N/A
N/A
7. Is an ANOVA or factorial design being used?
N/A
8. For data acquisition, is a whole brain scan used?
Yes
If not, state area of acquisition.
a. How was this region determined?
April 2015
6. How was behavioral performance measured?
N/A
7
a. Is the pulse sequence type (gradient/spin echo, EPI/spiral)

stated?
yes (3T; Online Methods section "Data")

yes; multiband EPI
b. Are the field-of-view, matrix size, slice thickness, and TE/TR/ yes
flip angle clearly stated?
10. Are the software and specific parameters (model/functions,

smoothing kernel size if applicable, etc.) used for data processing and
pre-processing clearly stated?
yes; more complete details of this public resource preprocessing are

in our papers referenced (Glasser, Smith, etc).
yes (grayordinates/MNI); see Online Methods throughout.

11. Is the coordinate space for the anatomical/functional imaging data
clearly defined as subject/native space or standardized stereotaxic
space, e.g., original Talairach, MNI305, ICBM152, etc? Where (section,
paragraph #)?
9. Is the field strength (in Tesla) of the MRI system stated?
yes; more complete details of this public resource preprocessing are

12. If there was data normalization/standardization to a specific space
in our papers referenced (Glasser, Smith, etc).
template, are the type of transformation (linear vs. nonlinear) used
and image types being transformed clearly described? Where (section,
paragraph #)?
13. How were anatomical locations determined, e.g., via an automated

labeling algorithm (AAL), standardized coordinate database (Talairach
daemon), probabilistic atlases, etc.?
N/A
14. Were any additional regressors (behavioral covariates, motion etc)

used?
yes; see full details in Online Methods section "CCA modeling".
15. Is the contrast construction clearly defined?
N/A
16. Is a mixed/random effects or fixed inference used?
N/A
a. If fixed effects inference used, is this justified?

17. Were repeated measures used (multiple measurements per subject)?
a. If so, are the method to account for within subject
correlation and the assumptions made about variance
clearly stated?
N/A
yes - 4 runs per subject
See Online Methods section "Network Matrices". 4 runs per subject
were combined before any cross-subject modeling, removing
within-subject autocorrelation.
18. If the threshold used for inference and visualization in figures varies, is N/A
this clearly stated?
a. If not, is this labeled as uncorrected?
yes
April 2015
19. Are statistical inferences corrected for multiple comparisons?
N/A
8
no
a. If so, is the rationale clearly described?
N/A
b. How were the ROIs defined (functional vs anatomical

localization)?
N/A
21. Is there correction for multiple comparisons within each voxel?
N/A
22. For cluster-wise significance, is the cluster-defining threshold and the

corrected significance level defined?
N/A
20. Are the results based on an ROI (region of interest) analysis?
Additional comments
Additional Comments
April 2015
9

Reporting Checklist For Nature Neuroscience: Statistics Reporting, by Figure

Uploaded by

Copyright:

Available Formats

Reporting Checklist For Nature Neuroscience: Statistics Reporting, by Figure

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Reporting Checklist For Nature Neuroscience: Statistics Reporting, by Figure

Uploaded by

Copyright:

Available Formats

Stephen Smith

nature neuroscience | reporting checklist