| Title: |
PAC-Bayesian Theorems for Domain Adaptation with Specialization to Linear Classifiers |
| Authors: |
Germain, Pascal; Habrard, Amaury; Laviolette, François; Morvant, Emilie |
| Contributors: |
Statistical Machine Learning and Parsimony (SIERRA); Département d'informatique - ENS-PSL (DI-ENS); École normale supérieure - Paris (ENS-PSL); Université Paris Sciences et Lettres (PSL)-Université Paris Sciences et Lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-École normale supérieure - Paris (ENS-PSL); Université Paris Sciences et Lettres (PSL)-Université Paris Sciences et Lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Centre National de la Recherche Scientifique (CNRS)-Centre Inria de Paris; Institut National de Recherche en Informatique et en Automatique (Inria); Laboratoire Hubert Curien (LabHC); Institut d'Optique Graduate School (IOGS)-Université Jean Monnet - Saint-Étienne (UJM)-Centre National de la Recherche Scientifique (CNRS); Département d'informatique et de génie logiciel Québec; Université Laval Québec (ULaval); NSERC discovery grant 262067; Université Jean Monnet, Saint-Étienne (42); Département d'Informatique et de Génie Logiciel, Université Laval (Québec); ENS Paris; IST Austria; ANR-09-CORD-0026,VideoSense,Reconnaissance multimodale de concepts enrichis (statiques, dynamiques, émotionnels) dans des vidéos multilingue au travers de langages pivots.(2009); ANR-09-EMER-0007,LAMPADA,Modèles et algorithmes d'apprentissage pour les données structurées et complexes(2009); European Project: 308036,ERC-2012-StG_20111012,ERC-2012-StG_20111012,L3VISU(2013) |
| Source: |
https://hal.science/hal-01134246 ; [Research Report] Université Jean Monnet, Saint-Étienne (42); Département d'Informatique et de Génie Logiciel, Université Laval (Québec); ENS Paris; IST Austria. 2016. |
| Publisher Information: |
CCSD |
| Publication Year: |
2016 |
| Collection: |
Université de Lyon: HAL |
| Subject Terms: |
Multisource; PAC-Bayesian Theory; Domain Adaptation; [STAT.ML]Statistics [stat]/Machine Learning [stat.ML] |
| Description: |
This report is a long version of our paper entitled A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classifiers published in the proceedings of the International Conference on Machine Learning (ICML) 2013. We improved our main results, extended our experiments, and proposed an extension to multisource domain adaptation ; In this paper, we provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different target distribution. On the one hand, we propose an improvement of the previous approach proposed by Germain et al. (2013), that relies on a novel distribution pseudodistance based on a disagreement averaging, allowing us to derive a new tighter PAC-Bayesian domain adaptation bound for the stochastic Gibbs classifier. We specialize it to linear classifiers, and design a learning algorithm which shows interesting results on a synthetic problem and on a popular sentiment annotation task. On the other hand, we generalize these results to multisource domain adaptation allowing us to take into account different source domains. This study opens the door to tackle domain adaptation tasks by making use of all the PAC-Bayesian tools. |
| Document Type: |
report |
| Language: |
English |
| Relation: |
info:eu-repo/semantics/altIdentifier/arxiv/1503.06944; info:eu-repo/grantAgreement//308036/EU/Life Long Learning for Visual Scene Understanding (L3ViSU)/L3VISU; ARXIV: 1503.06944 |
| Availability: |
https://hal.science/hal-01134246; https://hal.science/hal-01134246v3/document; https://hal.science/hal-01134246v3/file/main_report_PBDA.pdf |
| Rights: |
https://about.hal.science/hal-authorisation-v1/ ; info:eu-repo/semantics/OpenAccess |
| Accession Number: |
edsbas.4EBD555B |
| Database: |
BASE |