<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="sajjad-etal-2022-neuron">
    <titleInfo>
        <title>Neuron-level Interpretation of Deep NLP Models: A Survey</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hassan</namePart>
        <namePart type="family">Sajjad</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nadir</namePart>
        <namePart type="family">Durrani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Fahim</namePart>
        <namePart type="family">Dalvi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>The proliferation of Deep Neural Networks in various domains has seen an increased need for interpretability of these models. Preliminary work done along this line, and papers that surveyed such, are focused on high-level representation analysis. However, a recent branch of work has concentrated on interpretability at a more granular level of analyzing neurons within these models. In this paper, we survey the work done on neuron analysis including: i) methods to discover and understand neurons in a network; ii) evaluation methods; iii) major findings including cross architectural comparisons that neuron analysis has unraveled; iv) applications of neuron probing such as: controlling the model, domain adaptation, and so forth; and v) a discussion on open issues and future research directions.</abstract>
    <identifier type="citekey">sajjad-etal-2022-neuron</identifier>
    <identifier type="doi">10.1162/tacl_a_00519</identifier>
    <location>
        <url>https://aclanthology.org/2022.tacl-1.74/</url>
    </location>
    <part>
        <date>2022</date>
        <detail type="volume"><number>10</number></detail>
        <extent unit="page">
            <start>1285</start>
            <end>1303</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Neuron-level Interpretation of Deep NLP Models: A Survey
%A Sajjad, Hassan
%A Durrani, Nadir
%A Dalvi, Fahim
%J Transactions of the Association for Computational Linguistics
%D 2022
%V 10
%I MIT Press
%C Cambridge, MA
%F sajjad-etal-2022-neuron
%X The proliferation of Deep Neural Networks in various domains has seen an increased need for interpretability of these models. Preliminary work done along this line, and papers that surveyed such, are focused on high-level representation analysis. However, a recent branch of work has concentrated on interpretability at a more granular level of analyzing neurons within these models. In this paper, we survey the work done on neuron analysis including: i) methods to discover and understand neurons in a network; ii) evaluation methods; iii) major findings including cross architectural comparisons that neuron analysis has unraveled; iv) applications of neuron probing such as: controlling the model, domain adaptation, and so forth; and v) a discussion on open issues and future research directions.
%R 10.1162/tacl_a_00519
%U https://aclanthology.org/2022.tacl-1.74/
%U https://doi.org/10.1162/tacl_a_00519
%P 1285-1303

Markdown (Informal)

[Neuron-level Interpretation of Deep NLP Models: A Survey](https://aclanthology.org/2022.tacl-1.74/) (Sajjad et al., TACL 2022)

• Neuron-level Interpretation of Deep NLP Models: A Survey (Sajjad et al., TACL 2022)

ACL

• Hassan Sajjad, Nadir Durrani, and Fahim Dalvi. 2022. Neuron-level Interpretation of Deep NLP Models: A Survey. Transactions of the Association for Computational Linguistics, 10:1285–1303.