Biologically relevant transfer learning improves transcription factor binding prediction.
| Title: | Biologically relevant transfer learning improves transcription factor binding prediction. |
|---|---|
| Authors: | Novakovsky G; Centre for Molecular Medicine and Therapeutics, BC Children's Hospital Research Institute, Vancouver, BC, V5Z 4H4, Canada.; Department of Medical Genetics, University of British Columbia, Vancouver, BC, V6H 3 N1, Canada.; Saraswat M; Centre for Molecular Medicine and Therapeutics, BC Children's Hospital Research Institute, Vancouver, BC, V5Z 4H4, Canada.; Department of Medical Genetics, University of British Columbia, Vancouver, BC, V6H 3 N1, Canada.; Fornes O; Centre for Molecular Medicine and Therapeutics, BC Children's Hospital Research Institute, Vancouver, BC, V5Z 4H4, Canada. oriol@cmmt.ubc.ca.; Department of Medical Genetics, University of British Columbia, Vancouver, BC, V6H 3 N1, Canada. oriol@cmmt.ubc.ca.; Mostafavi S; Centre for Molecular Medicine and Therapeutics, BC Children's Hospital Research Institute, Vancouver, BC, V5Z 4H4, Canada.; Department of Medical Genetics, University of British Columbia, Vancouver, BC, V6H 3 N1, Canada.; Department of Statistics, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada.; Canadian Institute for Advanced Research, CIFAR AI Chair, and Child and Brain Development, Toronto, ON, M5G 1 M1, Canada.; Wasserman WW; Centre for Molecular Medicine and Therapeutics, BC Children's Hospital Research Institute, Vancouver, BC, V5Z 4H4, Canada. wyeth@cmmt.ubc.ca.; Department of Medical Genetics, University of British Columbia, Vancouver, BC, V6H 3 N1, Canada. wyeth@cmmt.ubc.ca. |
| Source: | Genome biology [Genome Biol] 2021 Sep 27; Vol. 22 (1), pp. 280. Date of Electronic Publication: 2021 Sep 27. |
| Publication Type: | Evaluation Study; Journal Article; Research Support, Non-U.S. Gov't |
| Language: | English |
| Journal Info: | Publisher: BioMed Central Ltd Country of Publication: England NLM ID: 100960660 Publication Model: Electronic Cited Medium: Internet ISSN: 1474-760X (Electronic) Linking ISSN: 14747596 NLM ISO Abbreviation: Genome Biol Subsets: MEDLINE |
| Imprint Name(s): | Publication: London, UK : BioMed Central Ltd; Original Publication: London : Genome Biology Ltd., c2000- |
| MeSH Terms: | Machine Learning*; Transcription Factors/*metabolism; Chromatin Immunoprecipitation Sequencing ; Genome |
| Abstract: | Background: Deep learning has proven to be a powerful technique for transcription factor (TF) binding prediction but requires large training datasets. Transfer learning can reduce the amount of data required for deep learning, while improving overall model performance, compared to training a separate model for each new task.; Results: We assess a transfer learning strategy for TF binding prediction consisting of a pre-training step, wherein we train a multi-task model with multiple TFs, and a fine-tuning step, wherein we initialize single-task models for individual TFs with the weights learned by the multi-task model, after which the single-task models are trained at a lower learning rate. We corroborate that transfer learning improves model performance, especially if in the pre-training step the multi-task model is trained with biologically relevant TFs. We show the effectiveness of transfer learning for TFs with ~ 500 ChIP-seq peak regions. Using model interpretation techniques, we demonstrate that the features learned in the pre-training step are refined in the fine-tuning step to resemble the binding motif of the target TF (i.e., the recipient of transfer learning in the fine-tuning step). Moreover, pre-training with biologically relevant TFs allows single-task models in the fine-tuning step to learn useful features other than the motif of the target TF.; Conclusions: Our results confirm that transfer learning is a powerful technique for TF binding prediction.; (© 2021. The Author(s).) |
| References: | Bioinformatics. 2009 Jun 1;25(11):1422-3. (PMID: 19304878); Nat Methods. 2020 Nov;17(11):1118-1124. (PMID: 33046896); Nat Rev Genet. 2016 Feb;17(2):93-108. (PMID: 26781813); Nucleic Acids Res. 2020 Jan 8;48(D1):D87-D92. (PMID: 31701148); Genome Res. 2015 Sep;25(9):1268-80. (PMID: 26160164); Nature. 2020 Jul;583(7818):693-698. (PMID: 32728248); Nucleic Acids Res. 2019 Jun 20;47(11):5735-5745. (PMID: 31114922); Bioinformatics. 2018 Jul 15;34(14):2483-2484. (PMID: 29514181); Nucleic Acids Res. 2020 Jan 8;48(D1):D756-D761. (PMID: 31691824); Nucleic Acids Res. 2012 Sep 1;40(17):e128. (PMID: 22610855); Genome Res. 2012 Sep;22(9):1798-812. (PMID: 22955990); BMC Genomics. 2014 Jun 13;15:472. (PMID: 24927817); Proc Natl Acad Sci U S A. 2013 Nov 12;110(46):18602-7. (PMID: 24173036); Nucleic Acids Res. 2016 Jun 20;44(11):e107. (PMID: 27084946); Genome Biol. 2014 Jul 29;15(7):412. (PMID: 25070602); Genome Biol. 2021 Sep 27;22(1):280. (PMID: 34579793); Nat Rev Genet. 2004 Apr;5(4):276-87. (PMID: 15131651); Nature. 2020 Sep;585(7825):357-362. (PMID: 32939066); Genome Biol. 2022 Jun 10;23(1):126. (PMID: 35681170); Brief Bioinform. 2017 Mar 1;18(2):279-290. (PMID: 26979602); Proc Natl Acad Sci U S A. 2020 Oct 13;117(41):25655-25666. (PMID: 32978299); Toxicology. 2002 Dec 27;181-182:131-41. (PMID: 12505298); Biochim Biophys Acta Gene Regul Mech. 2021 Nov-Dec;1864(11-12):194765. (PMID: 34673265); Trends Genet. 2015 Feb;31(2):67-76. (PMID: 25637093); Genome Biol. 2020 May 11;21(1):114. (PMID: 32393327); Nature. 2018 Oct;562(7725):76-81. (PMID: 30250250); PLoS Comput Biol. 2015 May 13;11(5):e1004220. (PMID: 25970446); Nature. 2012 Sep 6;489(7414):57-74. (PMID: 22955616); Nat Rev Genet. 2019 Jul;20(7):389-403. (PMID: 30971806); J Mol Biol. 2004 Apr 23;338(2):207-15. (PMID: 15066426); Bioinformatics. 2020 Mar 1;36(5):1553-1561. (PMID: 31608946); Nucleic Acids Res. 2017 Mar 17;45(5):e27. (PMID: 27899659); Nucleic Acids Res. 2018 Jan 4;46(D1):D267-D275. (PMID: 29126285); Trends Genet. 2020 Jul;36(7):523-539. (PMID: 32451166); Nucleic Acids Res. 2019 Jan 8;47(D1):D607-D613. (PMID: 30476243); Nucleic Acids Res. 2019 Feb 28;47(4):e21. (PMID: 30517703); Nat Methods. 2015 Oct;12(10):931-4. (PMID: 26301843); Nucleic Acids Res. 2018 Jan 4;46(D1):D343-D347. (PMID: 29087517); Genome Biol. 2019 Aug 12;20(1):165. (PMID: 31405383); J Biomol Struct Dyn. 2011 Dec;29(3):577-83. (PMID: 22066542); Nat Mach Intell. 2021 Feb;3(2):172-180. (PMID: 33796819); Int J Mol Sci. 2019 Jul 12;20(14):. (PMID: 31336830); Proc Natl Acad Sci U S A. 2021 Mar 9;118(10):. (PMID: 33658382); Bioinformatics. 2021 Sep 29;37(18):2834-2840. (PMID: 33760053); Nat Methods. 2019 Sep;16(9):875-878. (PMID: 31471617); Science. 2007 Jun 8;316(5830):1497-502. (PMID: 17540862); Cell Rep. 2016 Jun 14;15(11):2550-62. (PMID: 27264182); Nucleic Acids Res. 2020 Jan 8;48(D1):D180-D188. (PMID: 31665499); Genome Biol. 2007;8(2):R24. (PMID: 17324271); Bioinformatics. 2010 Mar 15;26(6):841-2. (PMID: 20110278); Genome Res. 2016 Jul;26(7):990-9. (PMID: 27197224); Nat Biotechnol. 2019 Jun;37(6):592-600. (PMID: 31138913); Curr Opin Syst Biol. 2020 Feb;19:16-23. (PMID: 32905524); PLoS Comput Biol. 2019 Sep 17;15(9):e1007324. (PMID: 31527870); Sci Rep. 2019 Dec 30;9(1):20353. (PMID: 31889137) |
| Grant Information: | PJT-162120 Canada CIHR |
| Contributed Indexing: | Keywords: Deep learning; Model interpretation; Transcription factor binding prediction; Transfer learning |
| Substance Nomenclature: | 0 (Transcription Factors) |
| Entry Date(s): | Date Created: 20210928 Date Completed: 20220128 Latest Revision: 20240403 |
| Update Code: | 20260130 |
| PubMed Central ID: | PMC8474956 |
| DOI: | 10.1186/s13059-021-02499-5 |
| PMID: | 34579793 |
| Database: | MEDLINE |
Evaluation Study; Journal Article; Research Support, Non-U.S. Gov't