qq_31367595的博客

2019全国知识图谱与语义计算大会前沿讲习班PPT和讲义！6_CCKS_ATT_ZhengdongLyu

2019全国知识图谱与语义计算大会前沿讲习班PPT和讲义！5_CCKS_ATT_YankaiLin

2019全国知识图谱与语义计算大会前沿讲习班PPT和讲义！4_CCKS_ATT_YongfengZhang

2019全国知识图谱与语义计算大会前沿讲习班PPT和讲义！3_CCKS_ATT_JiliangTang

Emerging Techniques for Semantic Search: Obvious Advantages, but ChallengesToo

知识图谱研究进展 - 自然语言处理视角，描述知识图谱领域最新的发展

认知概念图谱构建与应用，认知概念(Concept)是人类在认识过程中，从感性认识上升到理 性认识，把所感知的事物的共同本质特点抽象出来的一种表达。 认知概念不是非黑既白的，是认知场景下的符合实例本质的概率 分布。

来自达摩院智能服务的小蜜FAQ算法团队，关于FAQ-QA算法中台的思考和实践的报告

In relation extraction for knowledge-based question answering, searching from one entity to another entity via a single relation is called “one hop”. In related work, an exhaustivesearchfromallone-hoprelations,two-hop relations,andsoontothemax-hoprelationsin the knowledge graph is necessary but expensive. Therefore, the number of hops is generally restricted to two or three. In this paper, we propose UHop, an unrestricted-hop framework which relaxes this restriction by use of a transition-based search framework to replace the relation-chain-based search one. We conduct experiments on conventional 1- and 2hop questions aswell as lengthy questions, including datasets such as WebQSP, PathQuestion, and Grid World. Results show that the proposed framework enables the ability to halt, works well with state-of-the-art models, achieves competitive performance without exhaustive searches, and opens the performance gap for long relation paths.

Knowledge graphs are structured representations of real world facts. However, they typically contain only a small subset of all possible facts. Link prediction is a task of inferring missing facts based on existing ones. We propose TuckER, a relatively simple but powerful linear model based on Tucker decomposition of the binary tensor representation of knowledge graph triples. TuckER outperforms all previous state-of-the-art models acrossstandardlinkpredictiondatasets. Weprove that TuckER is a fully expressive model, deriving the bound on its entity and relation embedding dimensionality for full expressiveness which is several orders of magnitude smaller than the bound of previous state-of-the-art models ComplEx and SimplE. We further show that several previously introducedlinearmodelscanbeviewedasspecial cases of TuckER.

Knowledge graph embedding (KGE) is a technique for learning continuousembeddingsfor entities and relations in the knowledge graph. Due to its benefit to a variety of downstream tasks such as knowledge graph completion, question answering and recommendation, KGE has gained significant attention recently. Despite its effectiveness in a benign environment, KGE’s robustness to adversarialattacks is not well-studied. Existing attack methods on graph data cannot be directly applied to attack the embeddings of knowledge graph due to its heterogeneity. To fill this gap, we propose a collection of data poisoning attack strategies, which can effectively manipulate the plausibility of arbitrary targeted facts in a knowledge graph by adding or deleting facts on the graph. The effectiveness and efficiency of the proposed attack strategies are verified by extensive evaluations on two widely-used benchmarks.

Generating texts which express complex ideas spanning multiple sentences requires a structured representation of their content (document plan), but these representations are prohibitively expensive to manually produce.

Wikipedia tables represent an important resource, where informationisorganizedw.r.ttableschemasconsistingofcolumns.In turneachcolumn,maycontaininstance values thatpointtoother Wikipediaarticlesorprimitive values (e.g.numbers,stringsetc.).

Abstract. Knowledge graphs (KGs), i.e. representation of information as a semantic graph, provide a significant test bed for many tasks including question answering, recommendation, and link prediction. Various amount of scholarly metadata have been made available as knowledge graphs from the diversity of data providers and agents.

We study the problem of learning representations of entities and relations in knowledgegraphsforpredictingmissinglinks. Thesuccessofsuchataskheavily relies on the ability of modeling and inferring the patterns of (or between) the relations. Inthispaper,wepresentanewapproachforknowledgegraphembedding called RotatE, which is able to model and infer various relation patterns including: symmetry/antisymmetry,inversion,andcomposition. Specifically,theRotatE modeldefineseachrelationasarotationfromthesourceentitytothetargetentity inthecomplexvectorspace. Inaddition,weproposeanovelself-adversarialnegativesamplingtechniqueforefficientlyandeffectivelytrainingtheRotatEmodel. Experimentalresultsonmultiplebenchmarkknowledgegraphsshowthattheproposed RotatE model is not only scalable, but also able to infer and model various relation patterns and significantly outperform existing state-of-the-art models for link prediction.

Abstract—Security Analysts that work in a ‘Security Operations Center’ (SoC) play a major role in ensuring the security of the organization. The amount of background knowledge they have about the evolving and new attacks makes a significant difference in their ability to detect attacks. Open source threat intelligence sources, like text descriptions about cyber-attacks, can be stored in a structured fashion in a cybersecurity knowledge graph. A cybersecurity knowledge graph can be paramount in aiding a security analyst to detect cyber threats because it stores a vast range of cyber threat information in the form of semantic triples which can be queried. A semantic triple contains two cybersecurity entities with a relationship between them. In this work, we propose a system to create semantic triples over cybersecurity text, using deep learning approaches to extract possible relationships. We use the set of semantic triples generated through our system to assert in a cybersecurity knowledge graph. Security Analysts can retrieve this data from the knowledge graph, and use this information to form a decision about a cyber-attack.

Recently, there has been a surge of interest in learning representation of graphstructured data that are dynamically evolving. However, current dynamic graph learning methods lack a principled way in modeling temporal, multi-relational, and concurrent interactions between nodes—a limitation that is especially problematic for the task of temporal knowledge graph reasoning, where the goal is to predict unseen entity relationships (i.e., events) over time. Here we present Recurrent Event Network (RE-NET)—an architecture for modeling complex event sequences—which consists of a recurrent event encoder and a neighborhood aggregator. TheeventencoderemploysaRNNtocapture(subject,relation)-specific patterns from historical entity interactions; while the neighborhood aggregator summarizes concurrent interactions within each time stamp. An output layer is designed for predicting forthcoming, multi-relational events. Experiments1 on temporal link prediction over two knowledge graph datasets demonstrate the effectiveness of our method, especially on multi-step inference over time.

Graph-based methods have been demonstrated as one of the most effective approaches for semi-supervised learning,astheycanexploittheconnectivitypatternsbetweenlabeled and unlabeled data samples to improve learning performance. However, existing graph-based methods either are limited in their ability to jointly model graph structures and data features, such as the classical label propagation methods, or require a considerable amount of labeled data for training and validation due to high model complexity, such as the recent neural-network-based methods. In this paper, we address label efficient semi-supervised learning from a graph filtering perspective. Specifically, we propose a graph filtering framework that injects graph similarity into data features by taking them as signals on the graph and applying a low-pass graph filter to extract useful data representationsforclassification,wherelabelefficiencycan be achieved by conveniently adjusting the strength of the graph filter. Interestingly, this framework unifies two seemingly very different methods – label propagation and graph convolutional networks. Revisiting them under the graph filtering framework leads to new insights that improve their modelingcapabilitiesandreducemodelcomplexity. Experiments on various semi-supervised classification tasks on four citation networks and one knowledge graph and one semi-supervised regression task for zero-shot image recognition validate our findings and proposals.

To understand a scene in depth not only involves locating/recognizing individual objects, but also requires to infer the relationships and interactions among them. However, sincethedistributionofreal-worldrelationshipsisseriously unbalanced, existing methods perform quite poorly forthelessfrequentrelationships. Inthiswork,wefindthat the statistical correlations between object pairs and their relationships can effectively regularize semantic space and make prediction less ambiguous, and thus well address the unbalanced distribution issue. To achieve this, we incorporate these statistical correlations into deep neural networks to facilitate scene graph generation by developing a Knowledge-Embedded Routing Network. More specifically, weshowthatthestatisticalcorrelationsbetweenobjectsappearing in images and their relationships, can be explicitly representedbyastructuredknowledgegraph,andarouting mechanism is learned to propagate messages through the graph to explore their interactions. Extensive experiments on the large-scale Visual Genome dataset demonstrate the superiority of the proposed method over current state-ofthe-art competitors.

Generating long and semantic-coherent reports to describe medical images poses great challenges towards bridging visual and linguistic modalities, incorporating medical domain knowledge, and generating realistic and accurate descriptions. We propose a novel Knowledge-driven Encode, Retrieve, Paraphrase (KERP) approach which reconciles traditional knowledge- and retrieval-based methods with modern learning-based methods for accurate and robust medical report generation. Specifically, KERP decomposes medical reportgenerationintoexplicitmedicalabnormalitygraphlearning and subsequent natural language modeling. KERP first employs an Encode module that transforms visual features into a structured abnormality graph by incorporating prior medicalknowledge;thenaRetrievemodulethatretrievestext templates based on the detected abnormalities; and lastly, a Paraphrase module that rewrites the templates according to specificcases.ThecoreofKERPisaproposedgenericimplementation unit—Graph Transformer (GTR) that dynamically transforms high-level semantics between graph-structured data of multiple domains such as knowledge graphs, images andsequences.Experimentsshowthattheproposedapproach generates structured and robust reports supported with accurate abnormality description and explainable attentive regions, achieving the state-of-the-art results on two medical report benchmarks, with the best medical abnormality and disease classification accuracy and improved human evaluation performance.

Knowledge representation learning (KRL) aims to represent entities and relations in knowledge graph in low-dimensional semantic space, which have been widely used in massive knowledge-driven tasks. In this article, we introduce the reader to the motivationsforKRL,andoverviewexistingapproachesforKRL.Afterwards,weextensively conduct and quantitative comparison and analysis of several typical KRL methods on three evaluation tasks of knowledge acquisition including knowledge graph completion, triple classification, and relation extraction. We also review the real-world applicationsofKRL,suchaslanguagemodeling,questionanswering,informationretrieval, and recommender systems. Finally, we discuss the remaining challenges and outlook the future directions for KRL. The codes and datasets used in the experiments can be found in https://github.com/thunlp/OpenKE.

We present a family of novel methods for embedding knowledge graphs into real-valued tensors. These tensor-based embeddings capture the ordered relations that are typical in the knowledge graphs represented by semantic web languages like RDF. Unlike many previous models, our methods can easily use prior background knowledge provided by users or extracted automatically from existing knowledge graphs. In addition to providing more robust methods for knowledge graph embedding, we provide a provably-convergent, linear tensor factorization algorithm. We demonstrate the efficacy of our models for the task of predicting new facts across eight different knowledge graphs, achievingbetween5%and50%relativeimprovementoverexistingstate-of-the-artknowledgegraphembeddingtechniques. Our empirical evaluation shows that all of the tensor decomposition models perform well when the average degree of an entity in a graph is high, with constraint-based models doing better on graphs with a small number of highly similar relations and regularization-based models dominating for graphs with relations of varying degrees of similarity.

Two types of knowledge, triples from knowledge graphs and texts from unstructured documents, have been studied for knowledge aware open-domain conversation generation, in which triple attributes or graph paths can narrow down vertex candidates for knowledge selection decision, and texts can provide rich informationfor response generation. Fusion of a knowledge graph and texts might yield mutually reinforcing advantages for conversation generation, but there is less study on that. To address this challenge, we propose a knowledge aware chatting machine with three components, an augmented knowledge graph containingboth triples and texts, knowledgeselector, and response generator. For knowledgeselection on the graph, we formulate it as a problem of multi-hop graph reasoning that is more explainable and flexible in comparison with previous works. To fully leverage long text information that differentiates our graph from others, we improve a state of the art reasoning algorithm with machine reading comprehension technology. We demonstrate that supported by such unified knowledge and explainable knowledge selection method, our system can generate more appropriateand informative responses than baselines.

Toprovidemoreaccurate,diverse,andexplainablerecommendation, it is compulsory to go beyond modeling user-item interactions andtakesideinformationintoaccount.Traditionalmethodslike factorizationmachine(FM)castitasasupervisedlearningproblem, whichassumeseachinteractionasanindependentinstancewith side information encoded. Due to the overlook of the relations amonginstancesoritems(e.g., thedirectorofamovieisalsoan actorofanothermovie),thesemethodsareinsufficienttodistillthe collaborativesignalfromthecollectivebehaviorsofusers.

Explainabilityandeffectivenessaretwokeyaspectsforbuildingrecommendersystems.Prioreffortsmostlyfocusonincorporatingside informationtoachievebetterrecommendationperformance.However,thesemethodshavesomeweaknesses:(1)predictionofneural network-basedembeddingmethodsarehardtoexplainanddebug; (2)symbolic,graph-basedapproaches(e.g.,metapath-basedmodels) requiremanualeffortsanddomainknowledgetodefinepatterns andrules,andignoretheitemassociationtypes(e.g.substitutable andcomplementary).Inthispaper,weproposeanoveljointlearningframeworktointegrateinductionofexplainablerulesfromknowledgegraphwithconstructionofarule-guidedneuralrecommendation model. The framework encourages two modules to complement each other in generating effective and explainable recommendation:1)inductiverules,minedfromitem-centricknowledgegraphs, summarizecommonmulti-hoprelationalpatternsforinferringdifferentitemassociationsandprovidehuman-readableexplanation formodelprediction;2)recommendationmodulecanbeaugmented byinducedrulesandthushavebettergeneralizationabilitydealing with the cold-start issue. Extensive experiments1 show that our proposedmethodhasachievedsignificantimprovementsinitem recommendationoverbaselinesonreal-worlddatasets.Ourmodel demonstrates robust performance over “noisy" item knowledge graphs,generatedbylinkingitemnamestorelatedentities.

Reasoning is essential for the development of large knowledge graphs,especiallyforcompletion,whichaimstoinfernewtriples basedonexistingones.Bothrulesandembeddingscanbeusedfor knowledgegraphreasoningandtheyhavetheirownadvantages anddifficulties.Rule-basedreasoningisaccurateandexplainable butrulelearningwithsearchingoverthegraphalwayssuffersfrom efficiencyduetohugesearchspace.Embedding-basedreasoningis morescalableandefficientasthereasoningisconductedviacomputationbetweenembeddings,butithasdifficultylearninggood representationsforsparseentitiesbecauseagoodembeddingrelies heavilyondatarichness.Basedonthisobservation,inthispaper we explore how embedding and rule learning can be combined together and complement each other’s difficulties with their advantages.WeproposeanovelframeworkIterEiterativelylearning embeddingsandrules,inwhichrulesarelearnedfromembeddings with proper pruning strategy and embeddings are learned from existingtriplesandnewtriplesinferredbyrules.Evaluationson embeddingqualitiesofIterEshowthatruleshelpimprovethequality of sparse entity embeddings and their link prediction results. Wealsoevaluatetheefficiencyofrulelearningandqualityofrules fromIterEcomparedwithAMIE+,showingthatIterEiscapableof generatinghighqualityrulesmoreefficiently.

Representing entities and relations in an embedding space is a well-studied approach for machine learning on relational data. Existing approaches, however, primarily focus on improving accuracy and overlook other aspects such as robustness and interpretability. In this paper, we propose adversarial modifications for link prediction models: identifying the fact to add into or remove from the knowledge graph that changes the prediction for a target fact after the model is retrained. Using these single modifications of the graph, we identify the most influential fact for a predicted link and evaluate the sensitivity of the model to the addition of fake facts. We introduce an efficient approach to estimate the effect of such modifications by approximating thechangeintheembeddingswhentheknowledge graph changes. To avoid the combinatorial search over all possible facts, we train a network to decode embeddings to their corresponding graph components, allowing the use of gradient-based optimization to identify the adversarial modification. We use these techniquestoevaluatetherobustnessoflinkprediction models (by measuring sensitivity to additional facts), study interpretability through the factsmostresponsibleforpredictions(byidentifying the most influential neighbors), and detect incorrect facts in the knowledge base.

Abstract—Recent efforts show that neural networks are vulnerable to small but intentional perturbations on input features in visual classification tasks. Due to the additional consideration of connections between examples (e.g., articles with citation link tend to be in the same class), graph neural

Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plaintext,andbefine-tunedtoconsistentlyimprove the performance of various NLP tasks. However, the existing pre-trained language models rarely consider incorporating knowledge graphs (KGs), which can provide rich structuredknowledgefactsforbetterlanguage understanding. We argue that informative entities in KGs can enhance language representation with external knowledge. In this paper, we utilize both large-scale textual corpora and KGs to train an enhanced language representation model (ERNIE), which can take full advantage of lexical, syntactic, and knowledge information simultaneously. The experimental results have demonstrated that ERNIE achieves significant improvements on various knowledge-driven tasks, and meanwhile is comparable with the state-of-theart model BERT on other common NLP tasks. The source code of this paper can be obtained from https://github.com/ thunlp/ERNIE.

Abstract—Accurate retrieval of the power equipment information plays an important role in guiding the full-lifecycle management of power system assets. Because of data duplication, database decentralization, weak data relations, and sluggish data updates, the power asset management system eager to adopt a new strategy to avoid the information losses, bias, and improve the data storage efficiency and extraction process. Knowledge graph has been widely developed in large part owing to its schema-less nature. It enables the knowledge graph to grow seamlesslyandallowsnewrelationsadditionandentitiesinsertion when needed. This study proposes an approach for constructing power equipment knowledge graph by merging existing multisource heterogeneous power equipment related data. A graphsearch method to illustrate exhaustive results to the desired information based on the constructed knowledge graph is proposed. A case of a 500 kV station example is then demonstrated to show relevant search results and to explain that the knowledge graph can improve the efficiency of power equipment management

Abstract—The goal of knowledge representation learning is to embed entities and relations into a low-dimensional, continuous vector space. How to push a model to its limit and obtain better results is of great significance in knowledge graph's applications. We propose a simple and elegant method, Trans-DLR, whose main idea is dynamic learning rate control during training. Our method achieves remarkable improvement, compared with recent GAN-based method. Moreover, we introduce a new negative sampling trick which corrupts not only entities, but also relations, in different probabilities. We also develop an efficient way, which fully utilizes multiprocessing and parallel computing, to speed up evaluation of the model in link prediction tasks. Experiments show that our method is effective

In this paper we present, Cyber-All-Intel an artificial intelligence system to aid a security analyst. It is a system for knowledge extraction, representation and analytics in an end-toend pipeline grounded in the cybersecurity informatics domain. It uses multiple knowledge representations like, vector spaces and knowledge graphs in a ‘VKG structure’ to store incoming intelligence. The system also uses neural network models to proactively improve its knowledge. We have also created a query engine and an alert system that can be used by an analyst to find actionable cybersecurity insights.

中人工智能行业从CHAT_GPT到生成式AI（Generative+AI）：人工智能新范式，重新定义生产力

阿里小蜜

小规模标签Chaudhari-SPIGL2020

区块链综述，区块链技术

赛迪重磅发布《中国工业机器视觉产业发展白皮书》

2019全国知识图谱与语义计算大会前沿讲习班PPT和讲义！2_CCKS_ATT_XiangRen

2019全国知识图谱与语义计算大会前沿讲习班PPT和讲义！1_CCKS_ATT_XifengYan

2019全国知识图谱与语义计算大会前沿讲习班PPT和讲义！0_CCKS_ATT_Intro