What does 'Linear regularities among words' mean?












2












$begingroup$


Context: In the paper "Efficient Estimation of Word Representations in Vector Space" by T. Mikolov et al., the authors make use of the phrase: 'Linear regularities among words'.



What does that mean in the context of the paper, or in a general context related to NLP?



Quoting the paragraph from the paper:




Somewhat surprisingly, it was found that similarity of word
representations goes beyond simple syntactic regularities. Using a
word offset technique where simple algebraic operations are performed
on the word vectors, it was shown for example that vector(”King”) -
vector(”Man”) + vector(”Woman”) results in a vector that is closest to
the vector representation of the word Queen [20].



In this paper, we try to maximize accuracy of these vector operations
by developing new model architectures that preserve the linear
regularities among words. We design a new comprehensive test set for
measuring both syntactic and semantic regularities1 , and show that
many such regularities can be learned with high accuracy. Moreover, we
discuss how training time and accuracy depends on the dimensionality
of the word vectors and on the amount of the training data.







nlp language-model representation







share|improve this question









$endgroup$

















    2












    $begingroup$


    Context: In the paper "Efficient Estimation of Word Representations in Vector Space" by T. Mikolov et al., the authors make use of the phrase: 'Linear regularities among words'.



    What does that mean in the context of the paper, or in a general context related to NLP?



    Quoting the paragraph from the paper:




    Somewhat surprisingly, it was found that similarity of word
    representations goes beyond simple syntactic regularities. Using a
    word offset technique where simple algebraic operations are performed
    on the word vectors, it was shown for example that vector(”King”) -
    vector(”Man”) + vector(”Woman”) results in a vector that is closest to
    the vector representation of the word Queen [20].



    In this paper, we try to maximize accuracy of these vector operations
    by developing new model architectures that preserve the linear
    regularities among words    . We design a new comprehensive test set for
    measuring both syntactic and semantic regularities1 , and show that
    many such regularities can be learned with high accuracy. Moreover, we
    discuss how training time and accuracy depends on the dimensionality
    of the word vectors and on the amount of the training data.







    nlp language-model representation







    share|improve this question









    $endgroup$















      2












      2








      2





      $begingroup$


      Context: In the paper "Efficient Estimation of Word Representations in Vector Space" by T. Mikolov et al., the authors make use of the phrase: 'Linear regularities among words'.



      What does that mean in the context of the paper, or in a general context related to NLP?



      Quoting the paragraph from the paper:




      Somewhat surprisingly, it was found that similarity of word
      representations goes beyond simple syntactic regularities. Using a
      word offset technique where simple algebraic operations are performed
      on the word vectors, it was shown for example that vector(”King”) -
      vector(”Man”) + vector(”Woman”) results in a vector that is closest to
      the vector representation of the word Queen [20].



      In this paper, we try to maximize accuracy of these vector operations
      by developing new model architectures that preserve the linear
      regularities among words      . We design a new comprehensive test set for
      measuring both syntactic and semantic regularities1 , and show that
      many such regularities can be learned with high accuracy. Moreover, we
      discuss how training time and accuracy depends on the dimensionality
      of the word vectors and on the amount of the training data.







      nlp language-model representation







      share|improve this question









      $endgroup$




      Context: In the paper "Efficient Estimation of Word Representations in Vector Space" by T. Mikolov et al., the authors make use of the phrase: 'Linear regularities among words'.



      What does that mean in the context of the paper, or in a general context related to NLP?



      Quoting the paragraph from the paper:




      Somewhat surprisingly, it was found that similarity of word
      representations goes beyond simple syntactic regularities. Using a
      word offset technique where simple algebraic operations are performed
      on the word vectors, it was shown for example that vector(”King”) -
      vector(”Man”) + vector(”Woman”) results in a vector that is closest to
      the vector representation of the word Queen [20].



      In this paper, we try to maximize accuracy of these vector operations
      by developing new model architectures that preserve the linear
      regularities among words      . We design a new comprehensive test set for
      measuring both syntactic and semantic regularities1 , and show that
      many such regularities can be learned with high accuracy. Moreover, we
      discuss how training time and accuracy depends on the dimensionality
      of the word vectors and on the amount of the training data.







      nlp language-model representation




      nlp language-model representation






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 14 hours ago









      Dawny33♦Dawny33

      5,45183188




      5,45183188






















          1 Answer
          1






          active

          oldest

          votes


















          5












          $begingroup$

          By linear regularities among words, he meant that "Vectorized form of words should follow linear additive properties!"



          V("King") - V("Man") + V("Woman") ~ V("Queen)






          share|improve this answer









          $endgroup$













            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "557"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46648%2fwhat-does-linear-regularities-among-words-mean%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            5












            $begingroup$

            By linear regularities among words, he meant that "Vectorized form of words should follow linear additive properties!"



            V("King") - V("Man") + V("Woman") ~ V("Queen)






            share|improve this answer









            $endgroup$


















              5












              $begingroup$

              By linear regularities among words, he meant that "Vectorized form of words should follow linear additive properties!"



              V("King") - V("Man") + V("Woman") ~ V("Queen)






              share|improve this answer









              $endgroup$
















                5












                5








                5





                $begingroup$

                By linear regularities among words, he meant that "Vectorized form of words should follow linear additive properties!"



                V("King") - V("Man") + V("Woman") ~ V("Queen)






                share|improve this answer









                $endgroup$



                By linear regularities among words, he meant that "Vectorized form of words should follow linear additive properties!"



                V("King") - V("Man") + V("Woman") ~ V("Queen)







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered 13 hours ago









                PreetPreet

                2563




                2563






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Data Science Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f46648%2fwhat-does-linear-regularities-among-words-mean%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    "Incorrect syntax near the keyword 'ON'. (on update cascade, on delete cascade,)

                    Image
                    .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 2 total noob sorry any help greatly appreciated. I keep getting this "Incorrect syntax near the keyword 'ON'." heres my codes code one CREATE TABLE supplier ( supplierID INT PRIMARY KEY, ISBN INT FOREIGN KEY REFERENCES book(ISBN), price DECIMAL (5,2), ON UPDATE CASCADE, ON DELETE CASCADE, ); code two CREATE TABLE orders ( orderID INT PRIMARY KEY, customerID INT FOREIGN KEY REFERENCES customer(customerID), ISBN INT FOREIGN KEY REFERENCES book(ISBN), orderDate date, ON UPDATE CASCADE, ON DELETE CASCADE, ); Ive run out of stuff to try other than dumbing it down to uselessness. ...
                    Read more

                    Alcedinidae

                    Image
                    Alcedinidae Martin-pêcheur à dos bleu ( Alcedo azurea ) Classification (COI) Règne Animalia Embranchement Chordata Sous-embr. Vertebrata Classe Aves Ordre Coraciiformes Famille Alcedinidae Rafinesque, 1815 Les Alcédinidés ou Alcedinidae forment une famille d'oiseaux plus connus sous les noms de martins-pêcheurs et martins-chasseurs. Sommaire 1 Description 2 Répartition et habitat 3 Systématique 3.1 Liste alphabétique des genres 3.2 Liste des espèces 4 Chez les Anciens : l'alcyon 4.1 Étymologie (Littré) 4.2 Oiseau « fabuleux » ? 5 Notes 6 Articles connexes 7 Liens externes 7.1 Bibliographie Description | Ce sont des oiseaux compacts de taille petite à moyenne (10 à 46 cm), au bec droit et long, en forme de poignard. Leurs pattes sont courtes, et ils portent un plumage aux couleurs vives. Répartition et habitat | Cosmopolites, ils fréquentent surtout ...
                    Read more

                    Origin of the phrase “under your belt”?

                    Image
                    .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0; } 17 1 Today, I found myself discussing what students should have "under their belts" during a lecture, and I wondered to myself if there was some inappropriate undertone here that I might be unaware of. Question : What is the origin of the phrase "under your belt"? A google search revealed some webpages (e.g. [1]) that assert its origin is related to consumption of food -- once a meal has been eaten, it's under your belt (which is a relief). However, with such websites, I have no way to determine fact from "random guy on the internet making stuff up". etymology idioms ...
                    Read more