Scribd Logo
Upload

Welcome to Scribd!

  • Representation Learning On Networks: Stanford University

    Uploaded by

    Rafael Lima
    48 views17 pages
    This document discusses representation learning on networks. It introduces the field of learning embeddings or feature representations for nodes in networks. Networks are a general way to model complex systems across many domains like social networks, biological networks, and information networks. Representation learning aims to learn low-dimensional embeddings for nodes that can be used for downstream tasks like node classification, link prediction, and community detection. The document outlines challenges in applying traditional machine learning to graph-structured network data due to its complex topological structure and lack of fixed ordering. It previews discussing node embedding techniques and graph neural networks for deep learning on graphs.

    Original Description:

    Original Title

    nrltutorial-part0-intro

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document discusses representation learning on networks. It introduces the field of learning embeddings or feature representations for nodes in networks. Networks are a general way to model complex systems across many domains like social networks, biological networks, and information networks. Representation learning aims to learn low-dimensional embeddings for nodes that can be used for downstream tasks like node classification, link prediction, and community detection. The document outlines challenges in applying traditional machine learning to graph-structured network data due to its complex topological structure and lack of fixed ordering. It previews discussing node embedding techniques and graph neural networks for deep learning on graphs.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    48 views17 pages

    Representation Learning On Networks: Stanford University

    Uploaded by

    Rafael Lima
    This document discusses representation learning on networks. It introduces the field of learning embeddings or feature representations for nodes in networks. Networks are a general way to model complex systems across many domains like social networks, biological networks, and information networks. Representation learning aims to learn low-dimensional embeddings for nodes that can be used for downstream tasks like node classification, link prediction, and community detection. The document outlines challenges in applying traditional machine learning to graph-structured network data due to its complex topological structure and lack of fixed ordering. It previews discussing node embedding techniques and graph neural networks for deep learning on graphs.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 17

    Representation Learning

    on Networks
    Jure Leskovec, William L. Hamilton, Rex Ying, Rok Sosic
    Stanford University

    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 1


    Why networks?
    Networks are a general
    language for describing
    and modeling complex
    systems

    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 2


    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 3
    Network!
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 4
    Many Data are Networks

    A B

    Social networks Economic networks Biomedical networks

    Information networks: Figure 3: Higher-order cluster in the C. elegans neuronal network (28). A: The 4-node
    Internet Networks of neurons
    “bi-fan” motif, which is over-expressed in the neuronal networks (1). Intuitively, this motif
    Web & citations describes a cooperative propagation of information from the nodes on the left to the nodes on
    the right. B: The best higher-order
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWWcluster
    2018in the C. elegans frontal neuronal network
    5 based on
    Why Networks? Why Now?
    § Universal language for describing complex
    data
    § Networks from science, nature, and technology
    are more similar than one would expect
    § Shared vocabulary between fields
    § Computer Science, Social science, Physics,
    Economics, Statistics, Biology
    § Data availability (+computational challenges)
    § Web/mobile, bio, health, and medical
    § Impact!
    § Social networking, Social media, Drug design

    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 6


    Machine Learning with Networks
    Classical ML tasks in networks:
    § Node classification
    § Predict a type of a given node
    § Link prediction
    § Predict whether two nodes are linked
    § Community detection
    § Identify densely linked clusters of nodes
    § Network similarity
    § How similar are two (sub)networks
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 7
    Example: Node Classification

    ? ?

    ?
    ?
    Machine
    Learning
    ?

    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 8


    Example: Node Classification

    Classifying the
    function of
    proteins in the
    interactome!

    Image from: Ganapathiraju et al. 2016. Schizophrenia interactome with 504 novel
    protein–protein interactions. Nature.
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 9
    Example: Link Prediction

    ? x
    ? Machine
    Learning

    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 10


    Example: Link Prediction

    Content
    recommendation
    is link prediction! ?

    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 11


    Machine Learning Lifecycle
    § (Supervised) Machine Learning
    Lifecycle: This feature, that feature.
    Every single time!

    Raw Structured Learning


    Model
    Data Data Algorithm

    Feature Automatically Downstream


    Engineering learn the features prediction task
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 12
    Feature Learning in Graphs
    Goal: Efficient task-independent
    feature learning for machine learning
    in networks!

    node 2 vec
    u
    𝑓: 𝑢 → ℝ&
    ℝ&
    Feature representation,
    embedding
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 13
    Example
    § Zachary’s Karate Club Network:
    A B

    Input Output
    Image from: Perozzi et al. 2014. DeepWalk: Online Learning of Social
    Representations. KDD.
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 14
    Why Is It Hard?
    § Modern deep learning toolbox is
    designed for simple sequences or grids.
    § CNNs for fixed-size images/grids….

    § RNNs or word2vec for text/sequences…

    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 15


    Why Is It Hard?
    § But networks are far more complex!
    § Complex topographical structure
    (i.e., no spatial locality like grids)

    § No fixed node ordering or reference point


    (i.e., the isomorphism problem)
    § Often dynamic and have multimodal features.
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 16
    This talk
    § 1) Node embeddings
    § Map nodes to low-dimensional
    embeddings.

    § 2) Graph neural networks


    § Deep learning architectures for graph-
    structured data

    § 3) Applications
    Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 17

    You might also like