精华内容
下载资源
问答
  • The ability to build up view-invariant representations is central to a rich body of research on multiview learning. Given v1, v2 两个随机变量,本文阐述contrastive learning 就是 学习一个参数函数,鉴别 ...

    The ability to build up view-invariant representations is central to a rich body of research on multiview learning.

    Given v1, v2 两个随机变量,本文阐述contrastive learning 就是 学习一个参数函数,鉴别 sample是 来自于联合分布 p(v1)p(v2|v1) 还是说两个边缘分布的乘积 p(v1)p(v2). 一个区别是说前者的v2是基于v1产生的,可以理解为v1的正样本,后者v2与v1无关,可以理解为负样本。

    NCE LOSS:
    在这里插入图片描述

    原文提出了一个很有意思的假设:InfoMin假设。

    对比多视图学习的目标是学习参数编码器,其输出表示可用于区分具有相同标识的视图对和具有不同标识的视图对。视图之间共享的信息量和类型决定了结果模型在下游任务上的执行情况。我们假设产生最佳结果的视图应该在输入中丢弃尽可能多的信息,但与任务相关的信息(例如,对象标签)除外,这称为InfoMin原理。

    考虑下面的示例,其中同一图像的两个色块表示不同的“视图”。训练目标是识别两个视图属于同一图像。拥有过多信息的视图是不可取的,例如,低级颜色和纹理提示可被用作“快捷方式”(左),或者共享信息太少以至于无法识别它们属于图像的视图是不可取的。相同的图片(右)。相反,“最佳位置”上的视图共享与下游任务相关的信息,例如与对象分类任务(中心)的熊猫不同部分相对应的补丁。
    在这里插入图片描述
    解释一下就是下图
    在这里插入图片描述
    augmentation跟下游任务相关(保留下游任务的信息)可以帮助对比学习迁移到下游任务。augmentation 是 nuisance information 的时候,迁移到下游任务效果较差。
    在这里插入图片描述

    reference:
    https://ai.googleblog.com/2020/08/understanding-view-selection-for.html

    展开全文
  • 论文题目:What Makes Training Multi-modal Classification Networks Hard? 时间:2020 来源:CVPR 论文链接:点击跳转 论文代码:点击跳转 目录摘要简介 是什么让训练多模态分类网络变得困难? 摘要 考虑在具有...

    论文题目:What Makes Training Multi-modal Classification Networks Hard?
    时间:2020
    来源:CVPR
    论文链接:点击跳转
    论文代码:点击跳转


    是什么让训练多模态分类网络变得困难?

    摘要

    研究目的

      理论上多模态网络接收更多信息,因此它应该匹配或优于其单模态对应物。然而实验中,在不同模式组合以及不同任务和视频分类基本都观察到了相反的情况:最好的单模网络通常优于多模网络。

    问题原因

    导致这种性能下降的两个主要原因:

    1. 多模态网络由于容量增加通常容易过度拟合
    2. 不同的模态以不同的速率过拟合和泛化,因此用单一优化策略联合训练它们是次优的。

    解决问题

    提出梯度混合的技术来解决这两个问题,该技术根据它们的过度拟合行为计算模态的最佳混合。

    实验结果

      我们证明了梯度混合在避免过度拟合方面优于广泛使用的baseline,并在包括人类动作识别、以自我为中心的动作识别和声学事件检测在内的各种任务上实现了最先进的准确性。

    简介

      问题似乎是过拟合的:多模态网络具有更高的训练精度和更低的验证精度。后期视听(A+RGB)融合网络的参数几乎是单模态网络的两倍,过度拟合可能是由参数数量增加引起的。

    现有两种方法可以解决这个问题:

    1. 可以考虑诸如 dropout预训练提前停止等解决方案以减少过拟合
    2. 架构缺陷,我们通过串联门控融合进行中级融合实验,尝试了 Squeeze-and-Excitation (SE)门和非局部 (NL) 门。

      值得注意的是,这些都没有提供有效的解决方案。对于每种方法,我们在图 1 中记录了 Kinetics 上的最佳视听结果。预训练无法提供改进,并且提前停止往往会欠拟合 RGB模态。 Mid-concat 和 dropout 仅对 RGB 模型提供适度的改进。我们注意到 dropout 和 mid-concat(与后连接相比,参数减少 37%)比后连接提高了 1.5% 和 1.4%,证实了后连接的过度拟合问题。

      我们如何将这些实验与以前的多模式成功相协调?多模态网络已经成功地在包括声音定位 [59]、图像-音频对齐 [5] 和视听同步 [37, 34] 在内的任务上进行了联合训练。然而,这些任务不能用单一模态来执行,因此没有单模态基线,本文中发现的性能下降不适用。在其他工作中,通过使用预训练的单模态特征完全避免了联合训练。好的例子包括用于视频分类 [41, 49, 19, 12] 和图像文本分类 [6, 31] 的双流网络。这些方法不会联合训练多种模态,因此它们也没有可比性,并且由于独立训练,它们的准确性可能不是最佳的。

    我们在本文中的贡献包括:
    • 我们凭经验证明了多模态网络联合训练中过度拟合的重要性,并确定了导致该问题的两个原因。我们展示了这个问题是架构不可知的:不同的融合技术也可能遭受同样的过拟合问题。
    • 我们提出了一个度量来定量地理解问题:过度拟合泛化比 (OGR),具有理论和经验的依据。
    • 我们提出了一种新的训练方案,该方案通过多个监督信号的最佳混合(在某种意义上,我们在下面进行了精确)最小化了 OGR。这种 GradientBlending (G-Blend) 方法在消融方面取得了显着的进步,并通过结合音频和视觉信号在包括 Kinetics、EPIC-Kitchen 和 AudioSet 在内的基准测试中实现了最先进的 (SoTA) 精度。

      我们测试G-Blendisask不可知、架构不可知,并适用于其他场景(例如,在[39]中用于将点云与RGB相结合以进行3D对象检测)

    展开全文
  • Robert Waldinger: What makes a good life? Lessons from the longest study on happiness | TED Talk

    TED Talk URL for This Video:

    Robert Waldinger: What makes a good life? Lessons from the longest study on happiness | TED Talk

    Extensive Listening

    generated by python script

    English Script

    00:35

    What keeps us healthy and happy as we go through life? If you were going to invest now in your future best self, where would you put your time
    and your energy? There was a recent survey of millennials1 asking them what their
    most important life goals were, and over 80 percent said that a major life goal for them
    was to get rich. And another 50 percent
    of those same young adults said that another major life goal was to become famous.

    00:38

    (Laughter)

    01:21

    And we’re constantly told
    to lean in to work, to push harder and achieve more. We’re given the impression that these
    are the things that we need to go after in order to have a good life. Pictures of entire lives, of the choices that people make
    and how those choices work out for them, those pictures
    are almost impossible to get. Most of what we know about human life we know from asking people
    to remember the past, and as we know, hindsight2 is anything but 20/20. We forget vast amounts
    of what happens to us in life, and sometimes memory
    is downright creative.

    01:38

    But what if we could watch entire lives as they unfold through time? What if we could study people
    from the time that they were teenagers all the way into old age to see what really keeps people
    happy and healthy?

    02:10

    We did that. The Harvard Study of Adult Development may be the longest study
    of adult life that’s ever been done. For 75 years, we’ve tracked
    the lives of 724 men, year after year, asking about their work,
    their home lives, their health, and of course asking all along the way
    without knowing how their life stories were going to turn out.

    02:59

    Studies like this are exceedingly3rare. Almost all projects of this kind
    fall apart within a decade because too many people
    drop out of the study, or funding for the research dries up, or the researchers get distracted, or they die, and nobody moves the ball
    further down the field
    . But through a combination of luck and the persistence
    of several generations of researchers, this study has survived. About 60 of our original 724 men are still alive, still participating in the study, most of them in their 90s. And we are now beginning to study the more than 2,000 children of these men. And I’m the fourth director of the study.

    03:35

    Since 1938, we’ve tracked the lives
    of two groups of men. The first group started in the study when they were sophomores
    at Harvard College. They all finished college
    during World War II, and then most went off
    to serve in the war. And the second group that we’ve followed was a group of boys
    from Boston’s poorest neighborhoods, boys who were chosen for the study specifically because they were
    from some of the most troubled and disadvantaged families in the Boston of the 1930s. Most lived in tenements,
    many without hot and cold running water4.

    04:18

    When they entered the study, all of these teenagers were interviewed. They were given medical exams. We went to their homes
    and we interviewed their parents. And then these teenagers
    grew up into adults who entered all walks of life5. They became factory workers and lawyers
    and bricklayers and doctors, one President of the United States. Some developed alcoholism.
    A few developed schizophrenia6Some climbed the social ladder from the bottom
    all the way to the very top
    7, and some made that journey
    in the opposite direction.

    04:44

    The founders of this study would never in their wildest dreams have imagined that I would be
    standing here today, 75 years later, telling you that
    the study still continues. Every two years, our patient
    and dedicated research staff calls up our men
    and asks them if we can send them yet one more set of questions
    about their lives.

    04:56

    Many of the inner city Boston men ask us, “Why do you keep wanting to study me?
    My life just isn’t that interesting.” The Harvard men never ask that question.

    04:59

    (Laughter)

    05:35

    To get the clearest picture
    of these lives, we don’t just send them questionnaires8. We interview them in their living rooms. We get their medical records
    from their doctors. We draw their blood, we scan their brains, we talk to their children. We videotape9 them talking with their wives
    about their deepest concerns. And when, about a decade ago,
    we finally asked the wives if they would join us
    as members of the study, many of the women said,
    “You know, it’s about time.”

    05:38

    (Laughter)

    06:04

    So what have we learned? What are the lessons that come
    from the tens of thousands of pages of information that we’ve generated on these lives? Well, the lessons aren’t about wealth
    or fame or working harder and harder. The clearest message that we get
    from this 75-year study is this: Good relationships keep us
    happier and healthier
    . Period.

    07:01

    We’ve learned three big lessons
    about relationships. The first is that social connections
    are really good for us, and that loneliness kills
    . It turns out that people
    who are more socially connected to family, to friends, to community, are happier, they’re physically healthier,
    and they live longer than people who are less well connected
    . And the experience of loneliness
    turns out to be toxic10 People who are more isolated11 than they want to be from others find that they are less happy, their health declines earlier in midlife, their brain functioning declines sooner and they live shorter lives than people who are not lonely. And the sad fact
    is that at any given time, more than one in five Americans
    will report that they’re lonely.

    07:41

    And we know that you
    can be lonely in a crowd and you can be lonely in a marriage, so the second big lesson that we learned is that it’s not just
    the number of friends you have, and it’s not whether or not
    you’re in a committed relationship, but it’s the quality
    of your close relationships that matters
    . It turns out that living in the midst12 of conflict is really bad for our health.
    High-conflict marriages, for example,
    without much affection, turn out to be very bad for our health,
    perhaps worse than getting divorced. And living in the midst of good,
    warm relationships is protective.
    13

    08:48

    Once we had followed our men
    all the way into their 80s, we wanted to look back at them at midlife and to see if we could predict who was going to grow
    into a happy, healthy octogenarian14 and who wasn’t. And when we gathered together
    everything we knew about them at age 50, it wasn’t their middle age
    cholesterol15 levels that predicted how they
    were going to grow old. It was how satisfied they were
    in their relationships. The people who were the most satisfied
    in their relationships at age 50 were the healthiest at age 80. And good, close relationships
    seem to buffer16 us from some of the slings17 and arrows
    of getting old.
    18 Our most happily partnered men and women reported, in their 80s, that on the days
    when they had more physical pain, their mood stayed just as happy. But the people who were
    in unhappy relationships, on the days when they
    reported more physical pain, it was magnified19 by more emotional pain.

    09:44

    And the third big lesson that we learned
    about relationships and our health is that good relationships
    don’t just protect our bodies, they protect our brains.
    It turns out that being
    in a securely attached relationship to another person in your 80s
    is protective, that the people who are in relationships where they really feel they can count
    on the other person in times of need, those people’s memories
    stay sharper longer. And the people in relationships where they feel they really
    can’t count on the other one, those are the people who experience
    earlier memory decline. And those good relationships,
    they don’t have to be smooth all the time. Some of our octogenarian couples
    could bicker20 with each other day in and day out21 , but as long as they felt that they
    could really count on the other when the going got tough, those arguments didn’t take a toll22
    on their memories.

    11:04

    So this message, that good, close relationships
    are good for our health and well-being, this is wisdom that’s as old as the hills23 . Why is this so hard to get
    and so easy to ignore? Well, we’re human. What we’d really like is a quick fix, something we can get that’ll make our lives good
    and keep them that way. Relationships are messy
    and they’re complicated and the hard work of tending
    to family and friends, it’s not sexy or glamorous24 It’s also lifelong. It never ends. The people in our 75-year study
    who were the happiest in retirement were the people who had actively worked
    to replace workmates with new playmates. Just like the millennials
    in that recent survey, many of our men when they
    were starting out as young adults really believed that fame and wealth
    and high achievement were what they needed to go after
    to have a good life. But over and over, over these 75 years,
    our study has shown that the people who fared the best were
    the people who leaned in to relationships, with family, with friends, with community.

    11:15

    So what about you? Let’s say you’re 25,
    or you’re 40, or you’re 60. What might leaning in
    to relationships even look like?

    11:48

    Well, the possibilities
    are practically endless. It might be something as simple
    as replacing screen time with people time or livening up a stale relationship
    by doing something new together, long walks or date nights, or reaching out to that family member
    who you haven’t spoken to in years, because those all-too-common family feuds take a terrible toll on the people who hold the grudges.

    12:17

    I’d like to close with a quote
    from Mark Twain. More than a century ago, he was looking back on his life, and he wrote this: “There isn’t time, so brief is life, for bickerings, apologies,
    heartburnings, callings to account. There is only time for loving, and but an instant,
    so to speak, for that.”

    12:22

    The good life is built with good relationships.

    12:27

    Thank you.

    12:28

    (Applause)

    Chinese script

    00:35
    在我们的人生中 是什么让我们保持健康且幸福呢? 如果现在你可以 为未来的自己投资 你会把时间和精力投资在哪里呢? 最近在千禧一代中有这么一个调查 问他们生活中最重要的目标是什么 超过80%的人说 最大的生活目标就是要有钱 还有50%的年轻人说 另一个重要的生活目标 就是要出名

    00:38

    (笑声)

    01:21

    而且我们总是被灌输
    要投入工作,要加倍努力 要成就更多。 我们被灌输了这样一种观念,
    只有做到刚才说的这些 才能有好日子过。 要人们纵观整个人生, 想象各种选择,
    以及这些选择最终导致的结果, 几乎是不可能的。 关于人的一生,我们能了解到的, 大部分都是通过人的回忆得来, 但众所周知,大部分都是事后诸葛。 一生中,我们会忘记很多发生过的事情, 而且记忆常常不可靠。

    01:38

    但如果我们可以从头到尾地 纵观人的一生呢? 如果我们可以跟踪研究一个人,
    从他少年时代开始 一直到他步入晚年, 看看究竟是什么让人们
    保持快乐和健康呢?

    02:10

    我们做到了。 哈佛大学(进行的)这项
    关于成人发展的研究, 可能是同类研究中耗时最长的。 在75年时间里,
    我们跟踪了724个人的一生, 年复一年,了解他们的工作、
    家庭生活、健康状况, 当然,在这一过程中,
    我们完全不知道他们的人生 将走向何方。

    02:59

    像这样的研究少之又少。 像这样的项目几乎都会在10年内终止, 因为有许多人会中途退出, 或者是研究资金不足, 或者是研究者转换方向, 或者去世,然后项目无人接手。 但感谢幸运女神的眷顾 和几代研究人员的坚持不懈, 这个项目存活下来了。 目前这724人中 仍有60人在世, 仍然在参与研究 大多数人已经90多岁了。 现在我们已经开始研究 他们的子孙后代,
    人数多达2000多人。 我是这个项目的第四任负责人。

    03:35

    从1938年起,我们
    开始跟踪两组人的生活。 第一组加入这个项目的人, 当年在哈佛大学上大二。 他们在二战期间大学毕业, 大部分人都参军作战了。 我们追踪的第二组人 是一群来自波士顿贫民区的小男孩, 他们之所以被选中, 主要是因为他们来自
    20世纪30年代波士顿 最困难 最贫困的家庭。 大部分住在廉价公寓里,
    很多都没有冷热水供应。

    04:18

    在加入这个项目时, 这些年轻人都接受了面试。 接受了身体检查。 我们挨家挨户走访了他们的父母。 然后这些年轻人长大成人, 进入到社会各个阶层。 成为了工人、律师、砖匠、医生, 还有一位成了美国总统。 有人成为酒鬼,有人患了精神分裂。 有人从社会最底层 一路青云直上, 也有人恰相反,掉落云端。

    04:44

    这个项目的创始人们, 可能做梦都不会想到 75年后的今天,我会站在这里, 告诉你们这个项目还在继续。 每两年,我们耐心而专注的研究人员 会打电话给我们的研究对象,
    问他们是否愿意 再做一套关于他们生活的问卷。

    04:56

    那些来自波士顿的人问我们, “为什么你们一直想研究我?
    我的生活是很无趣的。” 但哈佛的人从没这样问过。

    04:59

    (笑声)

    05:35

    为了更好地了解这些人的生活, 我们不光给他们发问卷。 我们还在他们家客厅采访他们。 从他们医生那儿拿病历。 抽他们的血,扫描他们的大脑, 跟他们的孩子聊天。 我们拍摄下他们和妻子谈话的场景,
    聊的都是他们最关心的问题。 大约在10年前,我们
    终于开口问他们的妻子, 是否愿意加入我们的研究, 很多女士都说,“是啊,
    终于轮到我们了。”

    05:38

    (笑声)

    06:04

    那么我们得到了什么结论呢? 那长达几万页的数据记录,
    记录了他们的生活, 我们从这些记录中间, 到底学到了什么? 不是关于财富、名望,
    或更加努力工作。 从75年的研究中,
    我们得到的最明确的结论是: 良好的人际关系能
    让人更加快乐和健康。就这样。

    07:01

    关于人际关系,我们得到三大结论。 第一,社会关系对我们是有益的, 而孤独寂寞有害健康。 我们发现,那些跟家庭成员更亲近的人, 更爱与朋友、与邻居交往的人, 会比那些不善交际、离群索居的人, 更快乐,更健康,更长寿。 孤独寂寞是有害健康的。 那些“被孤立”的人,
    跟不孤单的人相比, 往往更加不快乐, 等他们人到中年时,健康状况下降更快, 大脑功能下降得更快, 也没那么长寿。 可惜的是,长久以来, 每5个美国人中就至少
    有1个声称自己是孤独的。

    07:41

    而且即便你身在人群中,
    甚至已经结婚了, 你还是可能感到孤独, 因此我们得到的第二大结论是 不是你有多少朋友, 也不是你身边有没有伴侣, 真正有影响的是这些关系的质量。 整天吵吵闹闹,对健康是有害的。 比如成天吵架,没有爱的婚姻, 对健康的影响或许比离婚还大。 而关系和睦融洽,
    则对我们的健康有益。

    08:48

    当我们的研究对象步入80岁时, 我们会回顾他们的中年生活 看我们能否预测 哪些人会在八九十岁时过得快乐健康 哪些人不会。 我们把他们50岁时的所有信息 进行汇总分析, 发现决定他们将如何老去的, 并不是他们中年时的胆固醇水平。 而是他们对婚姻生活的满意度。 那些在50岁时满意度最高的人, 在80岁时也是最健康的。 另外,良好和亲密的婚姻关系 能减缓衰老带来的痛苦。 参与者中那些最幸福的夫妻告诉我们, 在他们80多岁时, 哪怕身体出现各种毛病, 他们依旧觉得日子很幸福。 而那些婚姻不快乐的人, 身体上会出现更多不适, 因为坏情绪把身体的痛苦放大了。

    09:44

    关于婚姻和健康的关系,
    我们得到的第三大结论是, 幸福的婚姻不单能保护我们的身体, 还能保护我们的大脑。 研究发现,如果在80多岁时, 你的婚姻生活还温暖和睦, 你对自己的另一半 依然信任有加,
    知道对方在关键时刻能指望得上, 那么你的记忆力都不容易衰退。 而反过来, 那些觉得无法信任
    自己的另一半的人, 记忆力会更早表现出衰退。 幸福的婚姻,并不意味着从不拌嘴。 有些夫妻,八九十岁了, 还天天斗嘴, 但只要他们坚信,在关键时刻, 对方能靠得住, 那这些争吵顶多只是生活的调味剂。

    11:04

    所以请记住, 幸福和睦的婚姻对健康是有利的, 这是永恒的真理。 但为什么我们总是办不到呢? 因为我们是人类。 我们总喜欢找捷径, 总想一劳永逸, 找到一种方法,解决所有问题。 人际关系麻烦又复杂, 与家人、朋友相处需要努力付出, 一点也不高大上。 而且需要一辈子投入,无穷无尽。 在我们长达75年的研究中,
    那些最享受退休生活的人, 是那些主动用玩伴
    来替代工作伙伴的人。 就像开头我说过的千禧一代一样, 我们跟踪研究的很多人
    在年轻的时候 坚信名望、财富和成就 是他们过上好日子的保证。 但在75年的时间里,
    我们的研究一次次地证明, 日子过得最好的,
    是那些主动与人交往的人, 与家人、朋友或者邻居。

    11:15

    那么你们呢? 也许你现在25岁,
    或者40岁,或者60岁。 怎样才算主动与人交往呢?

    11:48

    嗯,我想有很多种方法吧。 最简单的,别再跟屏幕聊天了,
    去跟人聊天, 或者一起尝试些新事物,
    让关系恢复活力, 一起散个步呀,晚上约个会呀, 或者给多年未曾联系的亲戚打个电话, 因为这种家庭不和睦太常见了, 但它带来的伤害又很大, 尤其对那些喜欢
    生闷气的人来说更是如此。

    12:17

    我想引用马克•吐温的一段话来作为结束。 一个多世纪前, 他回首自己的人生, 写下这样一段话: “时光荏苒,生命短暂, 别将时间浪费在
    争吵、道歉、伤心和责备上。 用时间去爱吧, 哪怕只有一瞬间,也不要辜负。”

    12:22

    美好人生,从良好的人际关系开始。

    12:27

    谢谢大家。

    12:28

    (掌声)


    My Abstract

    generated by python script

    Footnote


    1. 英: {mɪ’leniəl} adj. 一千年的;一千年至福的;网络 千禧世代 ↩︎

    2. 英: 'haɪn(d).saɪt n. 事后聪明;事后的领悟; 网络:后见之明;蓝盐;表尺 ↩︎

    3. 英: [ɪkˈsiːdɪŋli] adv. 极其;非常;很 ↩︎

    4. 冷热自来水 ↩︎

    5. un. 〔外交〕各界;各行各业 ↩︎

    6. 英: [ˌskɪtsəʊˈfriːniə]
      n. 精神分裂症;早发性痴呆
      . ↩︎

    7. 有的从社会底层一路爬升到上流社会。 ↩︎

    8. n. 调查问卷;调查表 ↩︎

    9. v. 给…录像 ↩︎

    10. [ˈtɒksɪk]
      n. 毒物;毒剂. ↩︎

    11. [ˈaɪsəˌleɪtɪd] v. 分离 adj. 偏远的;孤立的;孤独的 ↩︎

    12. n. 中间;中部 ↩︎

    13. 而关系和睦融洽,则对我们的健康有益 ↩︎

    14. n. 八旬老人;80 至 89 岁的人 ↩︎

    15. n. 胆固醇 ↩︎

    16. v. 缓存;保护;减缓(伤害) ↩︎

    17. n. 吊索;抛;弹弓;吊环;【航】钩索;一关;【医】悬带;【军】(枪的)背带 ↩︎

    18. 另外,良好和亲密的婚姻关系 能减缓衰老带来的痛苦。 ↩︎

    19. v. 放大;夸大;夸奖 ↩︎

    20. n. 斗嘴;(流水等)哗啦哗啦的声响;(雨的)淅淅沥沥声;(鸟的)啭鸣
      v. 争吵 ↩︎

    21. 天天;夜以继日 ↩︎

    22. 产生负面影响;造成损失;产生严重的不良影响 ↩︎

    23. 这是亘古不变的智慧 ↩︎

    24. 英: [ˈɡlæmərəs]
      adj. 特别富有魅力的;富于刺激的. ↩︎

    展开全文
  • 作者:Ran Margolin,Ayellet Tal,Lihi Zelnik-Manor 论文网址:What Makes a Patch Distinct? | IEEE Conference Publication | IEEE Xplore 论文利用PCA进行显著性目标检测,没有看太懂,仅供参考。 论文提出,...

    作者:Ran Margolin,Ayellet Tal,Lihi Zelnik-Manor

    论文网址:What Makes a Patch Distinct? | IEEE Conference Publication | IEEE Xplore

     

    论文利用PCA进行显著性目标检测,没有看太懂,仅供参考。

    论文提出,共同考虑图像的patterns,colors,  high-level cues and priors.

    第一,Pattern Distinctness

    论文认为,the non-distinct patches of a natural image are mostly concentrated in the high-dimensional space, while distinct patches are more scattered. 并用图3进行说明。可以看出,以L1为例,60%的non-distinct patches are within a distance of 0.1, 但只有 20% of the distinct patches are within this distance.

    所以,论文认为可以通过衡量一个patch到平均patch的距离来决定其是否显著(距离平均patch越远,显著性值越大)。用L1来表示平均patch为:

    但是,作者进一步提出,如果某个patch出现在两幅不同图像中,并且图像的平均patch相同,那么这个patch在两个图像中的判定结果是一样的,但是,两个图像的分布可能不同,此时这个patch在两幅图中的显著性值应该是不一致的,这就导致了误判 。k-nearest patches approach 也面临同样的问题。如图4所示。

    因此,作者提出应该考虑图像的分布,并通过PC实现这一过程,如果一个patch连接到平均patch的路径很长,则认为其是显著的。这一过程通过计算PCA坐标下的一个patch到平均patch的L1-norm实现。因此, pattern distinctness 可以表示为:

    至此,根据图像的pattern来判定patch的显著性。论文认为这样做的好处是能够在高分辨率下进行图像分析。

    第二,Color Distinctness

    论文认为虽然pattern distinctness 能够识别图像中unique patterns,但是对所有图像是不够的,如图7。因此,采用两步法来区分颜色。首先,利用SLIC进行超像素分割(在PCA中已经用到,忘提了),然后在CIELab颜色空间中利用L2-norm表示color distinctness。

    第三,Putting it all together

     

     最终的显著性值为:

    然后归一化至[0,1]. 

    在此基础上,为了细化结果,考虑image organization,(这部分没太看懂,省略)。论文最终的显著性map:

    即 a simple product between the distinctness map and the Gaussian weight map。

    实验结果为:

    论文提供了run-time结果:

    但是在《基于背景先验和目标信息的自底向上显著性检测方法研究》这篇博士论文中作者认为本篇论文的运行耗时。。。。。

    展开全文
  • 论文笔记:What makes instance discrimination good for transfer learning? 这是一篇很有意思的论文,站在监督/无监督/迁移学习的角度讨论的,不过在这篇论文的阅读之前需要一部分迁移学习的基础来帮助更好的理解...
  • What Makes for Good Views for Contrastive Learning 主要对比了几种常见的对比学习,为什么能取得比较好的效果。 在保持与下游任务相关的信息的同时,减少视图之间的MI,才能取得比较好的效果。 数据增强是一种...
  • What Makes for Good Views for Contrastive Learning ?
  • What Should Not Be Contrastive in Contrastive Learning 引入了一个对比学习框架,该框架不需要事先特定的,与任务相关的不变性的知识,模型学会捕捉通过构建单独的视觉表示的可变和不变因素嵌入到空间,除了扩充...
  • Extensive Listening The TED TALK talks about what we should do ,facing the inevitability of this kind of injustice and the centrality of good fortune. Intensive Listening English Script 00:04 Hello, ...
  • 斯坦福课程Knowledge Graphs-What is a Knowledge Graph?1. Introduction2. Knowledge Graph Definition3. Recent Applications of Knowledge Graphs3.1 Knowledge Graphs for organizing Knowledge over the ...
  • What is Dynamic Programming?

    2021-12-10 12:22:27
    Dynamic Programming (DP) is a programming paradigm that can systematically and efficiently explore all ... Throughout this explore card, you will gain a better understanding of what makes DP so powerful.
  • Children are always sweet but they can sometimes make you feel bitter. In this problem, you will see how Tintin, a five year’s old boy, creates trouble for his parents.... But what he does is not a
  • Unit 12 -The Discovery of What It Means to Be an American ...It is a complex fate to be an American, Henry James observed, and the principal discovery an American writer makes in Europe is jus.
  • What Makes Good In-Context Examples for GPT-3? 2021 2021.1.17 motivation: 根据相似度选择 In-Context, order 根据相似度排序 18. Calibrate Before Use: Improving Few-Shot Performance of Language Models ...
  • I learned about serif and san serif typefaces, about varying the amount of space between different letter combinations, about what makes great typography great. It was beautiful, historical, ...
  • What is Markdown

    2021-01-12 10:33:51
    What is Markdown? Markdown is a lightweight markup language that you can use to add formatting elements to plaintext text documents. Created by John Gruber in 2004, Markdown is now one of the world’s...
  • Everyone knows about 2FA, because it used in our life widely. Like DUO, and the text ...Two factors, means there are another factor which makes everything safer. But what is a “factor”? or more precise
  • In addition to securing user data in transit, HTTPS makes sites more trustworthy from a user's perspective. Many users won't notice the difference between an http:// and an https:// web address, but ...
  • In the next line, print the number of operations ≤ that the robot makes in your example. In the each of next lines print two integers and — indices of positions that the robot will swap on this ...
  • What are the differences between Solid State Relays and Electromechanical Relays (SSR vs EMR)? INTRODUCTION: Nowadays, in more and more fields, traditional electromagnetic relays are being gradually...
  • What is Azure?

    2021-03-19 19:57:38
    What is Azure? Azure is a continually expanding set of cloud services that help your organization meet your current and future business challenges. Azure gives you the freedom to build, manage, and ...
  • WHAT IS ISO 15118? A number of different protocols and standards govern communications between hardware and software in the electric vehicle (EV) charging ecosystem. One of them is ISO15118. ISO ...
  • 正文字数:8279 阅读时长:12分钟2020年欧洲计算机视觉会议(ECCV)于8月23日至28日在网上举行,由1360篇论文组成,包括104场orals,160场spotlights...
  • what life is about

    2021-05-04 13:12:17
    try to do the things that makes you calm gradually accumulate the knowledge and strength for your further development in the long term run it is so essential a thing if something makes me feel nervous...
  • What Is Idempotence?

    2021-01-12 07:36:48
    Idempotence Idempotence is a funky word that often hooks people. Idempotence is sometimes a confusing ... This makes them idempotent as well since multiple, identical requests will behave the same.
  • Question 6:What makes you the right candidate for this PhD? Do say: I’ve been interested in this topic since the final year of my undergraduate degree. This lead to my choice of Masters and helped ...
  • Web Workers in IE10: Background JavaScript Makes Web Apps Faster07/01/20114 minutes to readIn this articleResponsive Web pages result in happier consumers. With Web workers, Web applications are more ...
  • What is overload in PHP?

    2021-04-24 13:20:45
    , The method overload implemented by the above method is very troublesome , Because it makes a certain method or magic method very heavy , It needs to be a controller to schedule internal methods ...
  • Unit 4B - Our Way of Life Makes Us Miserable Our Way of Life Makes Us Miserable Erich Fromm Most Americans believe that our society of consumption-happy, fun-loving, jet-traveling people creates ...
  • Lesson 48 - What Every Writer Wants I have known very few writers, but those I have known, and whom I respected, confess at once that they have little idea where they are going when they first set ...

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 89,509
精华内容 35,803
关键字:

makeswhat