gclda.dataset module

Class and functions for dataset-related stuff.

class gclda.dataset.Dataset(dataset_label, data_directory, mask_file=None)

Bases: future.types.newobject.newobject

Class object for a gcLDA dataset.

A Dataset contains data needed to run gcLDA models. It can also be used to view dataset information and can be saved to a pickled file.

Parameters:

dataset_label : str

The name of the gcLDA dataset. Also the name of a subfolder in data_directory containing four files (word_indices.txt, word_labels.txt, peak_indices.txt, and pmids.txt) with the data needed to create the dataset.

data_directory : str

The path to the folder containing the data. Should contain a subdirectory named after dataset_label with files needed to generate a Dataset.

mask_file : str, optional

A brain mask file used to define the voxels included in the dataset. If not provided, the mask file used by Neurosynth will be used by default.

Attributes:

dataset_label : str

The name of the dataset.

mask_img : nibabel.Nifti1Image

A nifti object of the mask file.

word_labels : list of str

List of word-strings (wtoken_word_idx values are indices into this list).

pmids : list of int

List of PubMed IDs (i.e., studies) in dataset.

wtoken_doc_idx : list of int

List of document-indices for word-tokens.

wtoken_word_idx : list of int

List of word-indices for word-tokens.

ptoken_doc_idx : list of int

List of document-indices for peak-tokens x.

peak_vals : numpy.ndarray of int

A focus-x-3 array of X, Y, and Z coordinates of foci in dataset in stereotactic (generally MNI152) space.

Methods

display_dataset_summary()	View dataset summary.
load(filename)	Load a pickled Dataset instance from file.
save(filename)	Pickle the Dataset instance to the provided file.
view_doc_labels([n_pmids])	View first n_pmids PMIDs in dataset.
view_peak_indices([n_peak_indices])	View first n_peak_indices peak indices.
view_word_indices([n_word_indices])	View first n_word_indices word indices.
view_word_labels([n_word_labels])	View first n_word_labels words in dataset.

next

display_dataset_summary()

View dataset summary.

classmethod load(filename)

Load a pickled Dataset instance from file. If the filename ends with ‘z’, it will be assumed that the file is compressed, and gzip will be used to load it. Otherwise, it will be assumed that the file is not compressed.

Parameters:

filename : str

File with saved Dataset instance.

Returns:

dataset : gclda.dataset.Dataset

Loaded Dataset instance.

next()

save(filename)

Pickle the Dataset instance to the provided file. If the filename ends with ‘z’, gzip will be used to write out a compressed file. Otherwise, an uncompressed file will be created.

Parameters:

filename : str

Where to save Dataset instance.

view_doc_labels(n_pmids=1000)

View first n_pmids PMIDs in dataset.

view_peak_indices(n_peak_indices=100)

View first n_peak_indices peak indices.

view_word_indices(n_word_indices=100)

View first n_word_indices word indices.

view_word_labels(n_word_labels=1000)

View first n_word_labels words in dataset.

gclda.dataset.import_neurosynth(neurosynth_dataset, dataset_label, out_dir='.', counts_file=None, abstracts_file=None, email=None, vocabulary=None)

Transform Neurosynth’s data into gcLDA-compatible files.

This function produces four files (word_indices.txt, word_labels.txt, peak_indices.txt, and pmids.txt) in a specified folder (out_dir/dataset_label), using data from a Neurosynth dataset and associated abstracts. These four files are necessary for creating a Dataset instance and running gcLDA.

Parameters:

neurosynth_dataset : neurosynth.base.dataset.Dataset

A Neurosynth Dataset object containing data needed by gcLDA.

dataset_label : str

The name of the gcLDA dataset to be created. A folder will be created in out_dir named after the dataset and output files will be saved there.

out_dir : str, optional

Output directory. Parent folder of a new folder named after dataset_label where output files will be saved. By default, it uses the current directory.

counts_file : str, optional

A tab-delimited text file containing feature counts for the dataset. The first column is ‘pmid’, used for identifying articles. Other columns are features (e.g., unigrams and bigrams from Neurosynth), where each value is the number of times the feature is found in a given article. This file is different from the features.txt file provided by Neurosynth, as it should contain counts instead of tf-idf frequencies, but it should have the same format. Only one of counts_file, abstracts_file, and email needs to be specified.

abstracts_file : str, optional

A csv file containing abstracts of articles in the database. The first column is ‘pmid’, used for identifying articles. The second column is ‘abstract’ and contains the article’s abstract. The abstracts_file can be created using download_abstracts in the Neurosynth Python package. Only one of counts_file, abstracts_file, and email needs to be specified.

email : str, optional

A valid email address. If neither counts_file nor abstracts_file is provided, then import_neurosynth will attempt to download article abstracts using Neurosynth’s download_abstracts function. This calls PubMed to get PMIDs and abstracts, which requires an email address. Only one of counts_file, abstracts_file, and email needs to be specified.

vocabulary : list of str, optional

A list of terms to use as the vocabulary for a dataset. Only works if abstracts_file or email address is provided (but not if counts_file is used).