extended-abstract

Analyzing large-scale genomic data with cloud data lakes

Authors:

Ohad BirkAuthors Info & Claims

SYSTOR '23: Proceedings of the 16th ACM International Conference on Systems and Storage

Page 142

https://doi.org/10.1145/3579370.3594750

Published: 22 June 2023 Publication History

Get Access

Abstract

In recent years there is huge influx of genomic data and a growing need for its analysis, yet existing genomic databases do not allow easy accessibility. We developed a pipeline that continuously pre-processes raw human genetic data. The data is then stored in a cloud data lake and can be accessed via a simple and intuitive web service and API.

References

[1]

Adam Ameur, Ignas Bunikis, Stefan Enroth, and Ulf Gyllensten. 2014. CanvasDB: a local database infrastructure for analysis of targeted-and whole genome re-sequencing projects. Database 2014 (2014).

Google Scholar

[2]

Noam Hadar, Grisha Weintraub, Ehud Gudes, Shlomi Dolev, and Ohad Birk. in press. GeniePool: Genomic Database With Corresponding Annotated Samples Based On a Cloud Data Lake Architecture. Database (in press).

Google Scholar

[3]

Rasko Leinonen, Hideaki Sugawara, Martin Shumway, and International Nucleotide Sequence Database Collaboration. 2010. The sequence read archive. Nucleic acids research 39, suppl_1 (2010), D19--D21.

Google Scholar

[4]

Grisha Weintraub, Ehud Gudes, and Shlomi Dolev. 2021. Needle in a haystack queries in cloud data lakes. In EDBT/ICDT Workshops.

Google Scholar

Cited By

View all

Weintraub GHadar NGudes EDolev SBirk O(2024)GeniePool 2.0: advancing variant analysis through CHM13-T2T, AlphaMissense, gnomAD V4 integration, and variant co-occurrence queriesDatabase10.1093/database/baae1302024Online publication date: 27-Dec-2024
https://doi.org/10.1093/database/baae130

Index Terms

Analyzing large-scale genomic data with cloud data lakes
1. Applied computing
  1. Life and medical sciences
    1. Computational biology
      1. Computational genomics

Recommendations

Genomic data modeling
Special issue: Data management in bioinformatics

Researchers face many challenges in representing biological data, including: (1) inherent complexity of biological data, (2) domain knowledge barrier, (3) constantly evolving knowledge, and (4) lack of expert data-modeling skills. We have studied how to ...
Provable data transfer from provable data possession and deletion in cloud storage

With the rapid development of cloud storage, an increasing number of users prefer to store their data on the remote cloud to reduce the burden of maintaining the data by themselves. Since different cloud storage providers offer distinct quality of ...
Large scale features in DNA genomic signals
Special issue: Genomic signal processing

Complex representations of the nucleotides, codons and amino acids derived from the projection of the Genetic Code Tetrahedron on adequately oriented planes are presented. By converting the sequences of nucleotides and polypeptides into digital genomic ...

Comments

Information & Contributors

Information

Published In

SYSTOR '23: Proceedings of the 16th ACM International Conference on Systems and Storage

June 2023

168 pages

ISBN:9781450399623

DOI:10.1145/3579370

Chair:
Yosef Moatti,
General Chair:
Ofer Biran
IBM Research - Haifa, Israel
,
Program Chairs:
Yossi Gilad
Hebrew University of Jerusalem, Israel
,
Dejan Kostic
KTH Royal Institute of Technology, Sweden

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 June 2023

Check for updates

Author Tags

Qualifiers

Extended-abstract

Funding Sources

Israeli Council for Higher Education (CHE) via Data Science Research Center, Ben-Gurion University of the Negev, Israel

Conference

SYSTOR '23

Sponsor:

SIGOPS

SYSTOR '23: 16th ACM International Conference on Systems and Storage

June 5 - 7, 2023

Haifa, Israel

Acceptance Rates

SYSTOR '23 Paper Acceptance Rate 12 of 30 submissions, 40%;

Overall Acceptance Rate 108 of 323 submissions, 33%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
41
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Weintraub GHadar NGudes EDolev SBirk O(2024)GeniePool 2.0: advancing variant analysis through CHM13-T2T, AlphaMissense, gnomAD V4 integration, and variant co-occurrence queriesDatabase10.1093/database/baae1302024Online publication date: 27-Dec-2024
https://doi.org/10.1093/database/baae130

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Genomic data modeling

Provable data transfer from provable data possession and deletion in cloud storage

Large scale features in DNA genomic signals

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations