[Tensorflow] K-Means

군집분석

텐서플로우를 활용한 군집화

• 출처 : 텐서플로우 첫걸음(한빛 미디어-박해선)
• 지도학습 : 모든 데이터에 레이블이 존재함
• 비지도 학습 : 데이터에 레이블이 존재 하지 않음
• k-평균 알고리즘 : 데이터를 유사한 것끼리 구룹화 할때 사용하는 알고리즘
• 텐셔를 이용해 K-평균 알고리즘 예재를 풀어봅시다

1. 기본자료구조 : 텐서

• 텐서플로우는 텐서라는 기본 자료구조로 모든 데이터를 표현함
• 텐서는 동적 크기를 갖는 다차원 데이터 배열로 문자, 논리, 숫자 등의 정적 자료형을 가짐
• 각 텐서는 배열의 차원을 나타내는 랭크를 가짐
• 랭크가 1인 텐서는 벡터, 랭크가 2인 텐서는 행렬이라고 볼수 있음(랭크 0인경우 스칼라값)
• 텐서플로우 공식문서에서는 구조, 랭크, 차원번호라는 3가지 명칭사용

텐서플로우 자료형	파이썬자료형	설명
DT.FLOAT	tf.float32	32비트 실수
DT.INT16	tf.int16	16비트 정수
DT.INT32	tf.int32	32비트 정수
DT.INT64	tf.int64	64비트 정수
DT.STRING	tf.string	문자열
DT.Bool	tf.bool	불리언

• 텐서플로우는 텐서를 다루는 여러가지 변환함수를 제공함

함수	설명
tf.shape	텐서의 구조를 알아냄
tf.size	텐서의 크기를 알아냄
tf.rank	텐서의 랭크를 알아냄
tf.reshape	텐서의 원소는 유지하면서 텐서의 구조를 바꿈
tf.squeeze	텐서의 크기가 1인 차원을 삭제
tf.expand_dims	텐서에 차원을 추가
tf.slice	텐서의 일부분을 삭제
tf.split	텐서의 한차원을 기준으로 여러개의 텐서로 나눔
tf.tile	한텐서를 여러번 중복해서 늘려 텐서를 만듬
tf.concat	한차원을 기준으로 텐서를 이어붙임
tf.reverse	텐서의 지정된 차원을 역전시킴
tf.transpose	텐서를 전치함

2.데이터를 얻는 방법

• 1) 데이터 파일로 부터 얻기 : 텐서플로우 사이트 활용문서 참고(http://goo.gl/r3VnFg)
• 2) 변수와 상수
  • tf.constant():상수를 생성하는 함수
  • tf.variable():변수를 생성하는 함수
• 텐서를 생성하는 함수

함수	설명
tf.zeros_like	모든 원소를 0으로 초기화 한 텐서생성
tf.one_like	모든 원소를 1로 초기화 한 텐서 생성
tf.fill	주어진 스칼라 값으로 원소를 초기화한 텐서 생성

함수	설명
tf.random_normal	정규분포를 따르는 난수 생성
tf.random_uniform	균등분포를 따르는 난수 생성
tf.truncated_normal	정규분포를 따르는 난수 + 2표준편차 벗어난 값은 제거

3. K-평균알고리즘

• 데이터의 유사성을 바탕으로 주어진 데이터를 지정된 k개의 군집으로 그룹화함
• 알고리즘의 결과는 중심(centroid)라고 불리는 K개의 점
• 군집을 구성할때 직접 오차함수를 최소화하려면 많은 계산량을 필요로함
• 이런문제를 해결하기위해 휴리스틱한 방법으로 로컬최소값에 빠르게 수렴하게하는 기법들이 개발됨
• 가장 널리사용되는 방법은 iterative refinement기법
  • 1)초기단계 : K개 중심의 초기 집합을 결정
  • 2)할당단계 : 각 데이터를 가장 가까운 군집에 할당
  • 3)업데이트단계 :각그룹에 대해 새로운 중심을 계산
  • 초기값 : K개를 임의로 선택
  • 할당단계와 수정 단계는는 알고리즘이 수렴됬다고 간주할때까지 루프를 통해 반복

실습

• 2000개의 난수를 생성하되 두개의 정규분포에서 생성함

In [1]:

import numpy as np
num_points = 2000
vector_set = []

for i in xrange(num_points) :
    if np.random.random()>0.5 :
        vector_set.append([np.random.normal(0.0,0.9),
                         np.random.normal(0.0,0.9)])
    else :
        vector_set.append([np.random.normal(3.0,0.5),
                          np.random.normal(3.0,0.5)] )

In [2]:

vector_set

Out[2]:

[[0.32921449652037543, -0.6544163077686246],
 [3.5300981064821304, 2.972335514771935],
 [3.7404056811563517, 2.61338000106367],
 [0.47671679955596863, -0.8315493666049446],
 [-0.45452145248225184, 0.520523986942455],
 [2.773385837109206, 3.4699988035023743],
 [-1.1395720985700586, -0.5495339829252511],
 [-0.12328289317814896, 1.1134756546314855],
 [-2.0912243326609117, -1.0741034265260005],
 [3.675440894633118, 3.2713417868943986],
 [2.8525608642857296, 3.2034670569573986],
 [2.380733247746017, 3.617102787310963],
 [2.323685557375457, 2.888759692803737],
 [3.1856748427008323, 3.2982735422666534],
 [4.22829354990299, 3.4122961380889354],
 [0.6276087066345353, -0.13508440565332439],
 [2.1540711237094454, 3.285340721230483],
 [3.8184724004997355, 3.1703905667358656],
 [3.1762308957704013, 3.616439215616067],
 [0.3660176717575661, 0.3078587541509089],
 [3.3712649836327566, 3.1501620906044616],
 [3.661035669791014, 3.04775998745197],
 [2.527721435344315, 3.1614372933136234],
 [2.491156786204212, 2.836003725169818],
 [-0.5221202958858143, -0.5825529454696186],
 [-1.7965757052218547, -0.9159405611882914],
 [1.11337537913239, -0.3029872485722855],
 [2.6270854076048575, 3.221556360209315],
 [-0.768976417693361, -0.6676314990958006],
 [-0.9400540824347691, 1.2502268733824875],
 [2.9445045499005755, 2.099787428045555],
 [3.341608332809912, 2.347060037355715],
 [3.6647220516241177, 2.6808775677242656],
 [2.6925726241586427, 2.6860406874568428],
 [3.653550763067345, 3.396296673093218],
 [3.176866711275679, 3.0635493205563566],
 [-0.3652029668662953, -0.7597471228668767],
 [-1.0878142908278752, -0.8378929564614048],
 [2.112666562412, 3.509146205533175],
 [-1.6788318583148263, -0.33313497938522313],
 [0.061709989669519155, 0.5189077514529999],
 [-0.5045346580727286, -1.560590062193415],
 [2.173115990537448, 2.6800369894015175],
 [0.8158074202051477, 2.336671880668372],
 [-0.7104159116274453, 1.2850420327484486],
 [2.8990348784959035, 3.987878256496797],
 [0.6987968748546591, 0.2238065394251495],
 [0.18194018817355234, -0.05893466069954281],
 [3.2637847377763896, 3.1955781879598977],
 [-0.9251854346357951, -0.8130949507039477],
 [2.9803165856249962, 2.987054038294449],
 [2.6075925507076443, 3.3846832606584076],
 [2.19195475455812, 3.157197810037565],
 [-0.6730389318547674, 0.34020440158093124],
 [-0.17537035852010951, 0.5325935047658579],
 [2.849361170673385, 3.0861249880937858],
 [0.04602339198370343, -0.3244848333796841],
 [4.403921553160712, 3.2536163104176286],
 [3.2208479539084096, 2.9224537534638877],
 [-2.337262901873194, -0.8064612173193988],
 [-0.4504394193838755, -1.4955108236295842],
 [3.2066060149634232, 3.4117469214680147],
 [2.6966441848153533, 3.3325567697115996],
 [-0.43602441063470704, 0.5193305221329226],
 [3.118295152873476, 2.6955545004061343],
 [2.831404378416545, 2.296682372980155],
 [2.058283354219374, 0.4842311843504849],
 [3.1129825544338434, 1.9464636369529693],
 [2.6450636963997223, 3.2338480640156573],
 [-2.074710426038671, -0.3064110035820638],
 [3.593190247710264, 3.2448405758028587],
 [0.9149053195835674, 0.4645981979549968],
 [2.930031515989184, 3.293240619331092],
 [-1.1397291860585135, 1.802932338158385],
 [-0.5200601520071605, -0.03917386365777213],
 [3.14567194902069, 2.136292630954869],
 [2.2752303657934707, 3.2964200488157505],
 [-0.5221463742915626, 0.1272866437303547],
 [-0.7813561764722352, -1.2332835183612472],
 [-1.4627600631285442, 0.6613815165473229],
 [2.8515868784828515, 3.288787471016986],
 [3.203296082236316, 3.7118448792460255],
 [0.8890745470539623, -0.0029383150939767504],
 [3.2484358875324575, 2.9812936233669953],
 [2.8045341469747647, 2.984902603984258],
 [-1.0041470675595885, 1.104175013711848],
 [3.5322488951811715, 2.74325286836389],
 [3.308333637864349, 3.0659392796142737],
 [3.1980466585383414, 2.9366587445958836],
 [3.240983193431588, 2.8078561272860543],
 [2.9151813295581457, 2.672743983903035],
 [-0.834074951740277, 0.7477623954228834],
 [-0.7566478163313053, 0.5683354805227295],
 [-0.11618068025612031, -0.734881997897413],
 [0.14243381696936783, -1.0326135274951072],
 [1.601294642574652, -1.0723563507263578],
 [2.9495255293847342, 2.295398192274142],
 [1.0176102407290915, -1.104646305303461],
 [2.251705801671125, 3.362694587826055],
 [3.4630198245919632, 3.4034796771820184],
 [3.441699775611489, 3.484807772074458],
 [0.12435052628408241, -1.0474059891338832],
 [-0.7466384868919689, -0.7051284563736401],
 [2.7620283028801462, 2.097720150132327],
 [3.0791303754712596, 3.0367955542174516],
 [-0.02106771804291607, 0.2695048047777793],
 [-1.3945511653971054, 1.3298576230530625],
 [2.3718502762932228, 2.8037625145060465],
 [-1.3853103964528006, 0.8211687412870557],
 [3.5982672530947797, 3.5357468275012502],
 [-0.7503606687703485, 0.7472873694217588],
 [3.0967659534802916, 2.9781041521719245],
 [3.739217525505313, 2.89041538381035],
 [2.7086413371873883, 3.0713088792640773],
 [-0.6778132493810729, -0.20941470801575016],
 [2.706669512216723, 2.965158416327457],
 [3.392304153185616, 3.4507232813055935],
 [-1.242946014237881, 0.29324494859742667],
 [-1.5563984461598017, -0.049962663982868805],
 [3.0402207732984166, 1.886993920556424],
 [3.3270593687218764, 4.071296287071091],
 [1.0219982603914757, 1.0406377041894774],
 [-0.9892397339214647, 1.3945714504545],
 [-0.2001524572584539, 1.7442901926086436],
 [2.8628426112676673, 2.7978137195868693],
 [3.1613075359783833, 3.0272293522501537],
 [3.576757560818921, 3.8345210548779547],
 [3.3361686043395906, 2.515103909887922],
 [0.7178170746615743, -0.5357473585738497],
 [2.5562656290480197, 2.5877550692437126],
 [0.630963400113038, -0.04202224118543088],
 [0.7984356778813436, -0.20187045717502944],
 [3.5391900494305664, 2.8236148662440472],
 [3.6029684715953993, 3.8238101131578883],
 [-0.8247823993351546, -0.13772790070397767],
 [0.22133102481411682, -0.7002066971489664],
 [3.567867340475716, 3.893109474793456],
 [-0.24937131966466813, 0.0320552992085355],
 [0.9627700291602216, -0.7308947886962839],
 [2.6881805244593813, 2.12704500243849],
 [0.0964884544486375, 1.21550480893678],
 [3.3538179505086623, 2.0833096868540073],
 [2.5891448109850894, 3.0263468257933366],
 [-0.07276953705290656, 0.735283433937212],
 [3.0436765599614835, 2.2979920193729306],
 [-0.4478306701019, -0.06655431881520697],
 [3.465111658119171, 3.7623989772034188],
 [2.3370947888222355, 2.644347946989627],
 [2.9417423828901255, 2.2163163489219055],
 [3.4399689274235503, 2.9196530215705234],
 [0.8426814727985764, -0.20663672259003849],
 [3.05764412474959, 3.161112990303339],
 [-0.4869344034702456, -0.9669974355482805],
 [3.3849050322910466, 2.557902559518657],
 [3.319324521408365, 3.0964201128899496],
 [3.9207388867246338, 2.1353493801925882],
 [2.767405502317625, 2.6436619372886336],
 [-0.030417396890069608, 0.37905500750308985],
 [0.07148035781245524, -0.32257651521604586],
 [2.499363777548721, 2.1509893607300263],
 [-0.5141830649189054, -0.8023419958982432],
 [3.007721811637844, 3.7740816755273143],
 [0.4079455951671771, 1.1780967627683412],
 [0.6802809669767605, 0.582616378567116],
 [0.7116133724103059, 0.29021053840916],
 [0.8361504835195832, -0.6267406226048778],
 [3.2081553157613087, 3.323562304281366],
 [3.0897748553921165, 3.703611676815746],
 [1.191707266602997, -0.5428575626840648],
 [4.176753916252951, 1.7871983976451629],
 [2.5320658460850987, 2.845133984347663],
 [0.5277188205381725, 0.3478218214213864],
 [3.1324654811724884, 1.9865097116655253],
 [1.0479089635984562, 0.5577614274745378],
 [-0.8590756020260841, -1.1452285066194918],
 [3.292950924177264, 2.2296029681239444],
 [3.339685640834791, 2.8683749738005124],
 [-0.4996926286336418, 0.1069333115215717],
 [-0.4268772332731582, 0.3445663126502589],
 [3.8413260441350134, 2.995250154539368],
 [0.5274158921924496, 0.15044493222295593],
 [1.7166411955029182, -0.12155866861336428],
 [3.4468569221126666, 2.238404079514243],
 [2.4302127703700322, 3.6706410829949103],
 [2.7415825910150993, 3.3384976222547156],
 [-0.8813823216821242, -1.2498950114916636],
 [1.8380046989319359, 3.122095641361328],
 [0.9334279152873133, -2.646120096018777],
 [2.781063531025961, 2.344134602394376],
 [1.0893496662046203, -0.5145960562883806],
 [-0.5613660015403124, -0.8798619040651273],
 [2.6786130917263407, 2.540633162265172],
 [-0.6031656980784255, 0.7566899731684028],
 [3.970557074110622, 3.1027934930696377],
 [3.4354045424225976, 2.5124909073876815],
 [-2.3054692844265703, -0.8584250070315643],
 [3.7981024341697367, 2.427387183881853],
 [2.75010967782069, 3.4237562164459905],
 [0.1586388323720452, -0.8055889045123377],
 [2.5148117625103596, 3.0835314430353886],
 [1.4691313543475237, -0.2896476311346555],
 [0.3704472406567853, 0.6431252613172915],
 [2.421334112691157, 2.7568250552636626],
 [0.3563306339069519, -0.9205351853606519],
 [3.638395401075343, 3.9128413484574374],
 [-0.5972725578836271, -1.9463936734218945],
 [3.520741039954929, 3.038600689394155],
 [0.21150731046831248, -1.3932694404151178],
 [1.060050601431786, 0.10722188745396667],
 [-0.2608962589872447, -0.6694007254058862],
 [-0.4834117350534421, -0.11589772148998648],
 [2.7826300006114457, 3.0655473050916338],
 [-0.8857833434874737, 0.1457676733645938],
 [3.488796385381232, 3.2129038180994307],
 [4.071025591556991, 2.954143763447977],
 [2.514036679071487, -0.29435749527572014],
 [-1.3842064310333042, 0.2071770068395302],
 [0.3195495138392798, 0.07703480738779817],
 [3.7219465027581804, 3.390800975650909],
 [0.4136443635842806, 0.4721757818399557],
 [-0.32277566063802055, 0.22272387332896576],
 [2.0337943433183208, 2.877843842409056],
 [0.7036566655960416, 0.6598089630129529],
 [3.486644354167012, 3.765734594169546],
 [2.7523622790498674, 2.9079246148366633],
 [2.6782317025054687, 3.456892140183573],
 [3.25988502034499, 3.626574432118732],
 [2.6095222266780995, 2.119529652954425],
 [3.6823348286294313, 3.396838529770596],
 [-1.5485433702071778, 0.9764401628752083],
 [2.9745520277108595, 3.4948175187668875],
 [3.2149513314918963, 2.781909621586502],
 [3.148194970786515, 2.9865014204384286],
 [2.7079134376489353, 3.375614510190034],
 [3.251933139906285, 3.021193140399934],
 [4.081694142002108, 3.2214152024680582],
 [2.9857658941785843, 3.4807490680557023],
 [-0.6653135900434813, 1.078443572912592],
 [3.129337191153253, 3.2968292986747123],
 [-1.0629719501035153, 0.0025017552157880272],
 [2.6036951113436717, 3.1202728775625537],
 [3.2027955063347093, 2.8667810775584885],
 [-0.7673455644604013, -0.12938536008747356],
 [0.35231606614738065, 1.2646222005426881],
 [1.1288086364994898, 1.1462975170660579],
 [4.083561796262335, 2.5227083554944025],
 [2.7387184845469665, 3.9444087389209943],
 [1.3514366492441727, 0.47619187986756734],
 [0.4976336226938247, -1.6007356045970245],
 [2.6401737354535078, 3.437911052816104],
 [3.232028168761352, 3.418116248235008],
 [2.833533945282568, 3.2793331525392064],
 [3.9948627769518983, 3.3252950778436334],
 [2.6972894371554013, 2.6999705230687505],
 [0.49913171351745134, -0.15684127768131942],
 [3.0344199424795617, 2.2619499307480946],
 [-0.6258615590144754, 0.790162872403848],
 [-0.06903503455001708, -1.1793149878316627],
 [2.538505117635018, 2.564849252453382],
 [1.1040505499033408, -0.050767011313695064],
 [1.3770198088536258, 2.408130804372076],
 [-0.2554726669385601, -0.6032554847034023],
 [2.9150113376274747, 2.954074611558647],
 [-0.10527417410151808, 0.1860603208869616],
 [1.4044856444330067, -0.3093139047880305],
 [2.985640776442894, 2.0317453702506865],
 [1.9663817273046273, -0.7596825898091125],
 [3.297639616848211, 2.65544835785432],
 [0.8589612100983729, -0.3607381987107562],
 [0.17758184897036378, 0.653914046236301],
 [0.061495976692890905, -0.009693120920100204],
 [1.3856465811478074, -0.10561127828372605],
 [-0.38124047190692467, 1.021742789272849],
 [0.1745678367242228, -1.2055836492268086],
 [3.5413838344826436, 2.413441266439006],
 [-0.6593382139405429, 0.1818244634781652],
 [-0.7836184770587499, -0.6741266447012269],
 [3.170212496568749, 3.9151089040806886],
 [-1.4034758842624702, -1.1233173679162005],
 [0.1530179763075196, -0.5682209119210471],
 [0.1424594579321738, 0.49951020699767584],
 [2.8409811782062673, 2.792931632204332],
 [3.730638168057454, 2.863421496981131],
 [0.2249750424050286, -0.0028191557626964367],
 [3.2273100866264435, 2.2466739035602785],
 [2.7699757142581527, 2.4198816166144987],
 [-0.33949769372688393, 1.0149829847355312],
 [-0.11865278536480994, 0.40614967342739755],
 [3.54728720557993, 3.5733486934833625],
 [0.2545922100587207, 1.3205916536096391],
 [-1.8595402931622758, -0.754930420478041],
 [2.597087645329012, 3.2547322516006454],
 [-1.1155690094114352, -0.8722790987412189],
 [2.650462032497387, 3.5587873061855513],
 [0.9176902346014925, -1.1615094364197407],
 [0.41073488661143326, -0.5522291668749931],
 [2.1194397698054686, 3.3310562636709142],
 [-1.244685061994369, 0.19537062729746332],
 [0.22368090648419336, 0.07470086007043093],
 [-0.19263500881851142, 0.3238123172768327],
 [0.7133100767348656, -2.3935013126220808],
 [2.0301208787193867, 2.4592759043480865],
 [-0.5535036929568271, 0.44002033152552533],
 [0.9434098901923934, 0.7814422527508048],
 [-1.2754220082492305, -0.1620337561221051],
 [0.44781872468980105, 0.5786074465834646],
 [-1.1025485300736626, -1.2586955120058316],
 [0.20880225073024916, 0.8356438675723439],
 [0.4660605314156198, -0.09449607772947083],
 [2.756104040912962, 2.569883440787345],
 [0.9722840634599249, 1.5404904416623708],
 [0.448017335569309, -0.42505873705995434],
 [3.189665151036476, 3.8796791262348505],
 [3.307444222681041, 3.1482063538874216],
 [0.10899602854267973, -0.5243199622172912],
 [2.9351145863996817, 3.234098478368857],
 [3.385051575384772, 1.8825037687698984],
 [2.805474915098694, 0.7951585423925002],
 [-0.6647506641064791, -1.995701895323395],
 [3.7008560399129555, 2.9112165019157072],
 [1.2291925586306036, 1.498596367192278],
 [2.3994322970288047, 2.9871648578221874],
 [3.844948440669488, 3.0838494122563387],
 [2.209005080952943, 2.924960192717272],
 [2.7290402087831085, 2.7223924502452808],
 [3.2125537463357956, 3.395184256076539],
 [-1.358521215323136, 0.4287994091348964],
 [3.581373038139164, 2.6238108725696554],
 [-1.158969660117441, 0.5330498291186174],
 [3.047979863807251, 3.4079260652530565],
 [1.4146155940346195, -0.4850642555142334],
 [2.6260307245506076, 3.1794930840418254],
 [0.8202392014277305, 0.8244518851080951],
 [-1.1113725274317818, -1.0678739343375816],
 [0.8439042340404502, -0.5249460071488362],
 [0.9141686956089632, 0.15231247405521312],
 [2.6901889428167296, 3.3782044972580403],
 [0.5929873380468481, 0.24239176773661325],
 [1.372741859287853, -1.07723221062577],
 [3.389167577505941, 2.5003731452328943],
 [-0.4439667522071, 0.991901337108474],
 [-1.4098287994914713, -0.20088376176951378],
 [-0.20363623568984718, 0.45906371711245764],
 [-1.3834291638664293, 1.4798632240276088],
 [2.7841872783830954, 3.386481539801486],
 [1.098728737877298, 0.2148285177436564],
 [0.022251164945581912, -1.2997959438947697],
 [0.23696522044611992, -0.007228858930267515],
 [3.070872041040763, 4.018275859219874],
 [2.7761731802994176, 3.936954375565628],
 [3.190479745275259, 1.7945063266967374],
 [-1.1156064029357633, -0.5582972254560633],
 [3.3939244526682675, 2.5276202663323715],
 [0.6765956400250631, -1.035460075659284],
 [2.2779388858407588, 3.2630128424062024],
 [0.7084261197169293, 0.5736548734011784],
 [1.0641764622955843, -1.1177328454640123],
 [1.6265294910721808, 1.648799754866538],
 [-0.8023269655856041, 0.7489162270414609],
 [2.4273809964786537, 3.690949546538554],
 [2.5839541749009824, 3.077178797496854],
 [-1.1653936243945187, 0.002158285937802615],
 [3.5621174878953035, 3.1844757730566475],
 [3.1764585676674377, 2.9913811941981434],
 [3.1516606481883005, 3.6242709061263705],
 [3.3135168007569447, 3.4265371762368306],
 [-0.599976569497227, -0.1831757227279136],
 [2.4453433302582677, 2.7175095022900213],
 [0.6379449031918579, -0.5034422314543406],
 [3.8337706943597896, 3.063225722522923],
 [2.8292979655046935, 3.665656871187256],
 [3.662471535194712, 3.3232257340331985],
 [-1.356303007343041, 0.6383712559387106],
 [-0.027267956536804715, 0.8810185569417862],
 [3.918069689859443, 2.707756208139279],
 [3.070134241965973, 2.972313295503068],
 [0.609400915437573, -0.8162105088847634],
 [3.4671154174963057, 2.3270677715333283],
 [2.460316527433449, 3.033165298420787],
 [3.0614417531405604, 2.093883746229456],
 [3.909650300016806, 2.3061122985856275],
 [3.481206922712901, 2.9485852238274157],
 [0.5880351619514953, 0.22278479577049887],
 [3.3020990299083413, 2.7908403161122233],
 [-0.6747006739682305, -1.4628919587330333],
 [3.2916322502001423, 3.3255598259428205],
 [1.604326883707416, 1.1546622419392365],
 [2.9138266517094764, 3.3652299233026315],
 [-1.3430181342487022, -1.320515037081497],
 [-0.9925949012262967, 0.9717914508682131],
 [2.3503586687814964, 3.1534016644358096],
 [2.846295227571891, 2.237084274632523],
 [1.8441866764686552, -0.21420008393652032],
 [0.9016287579583859, -0.3000429087478779],
 [3.233304651715872, 3.1190586898276296],
 [-1.225664293710464, 0.05125020926949068],
 [2.867423327277338, 3.591372286321896],
 [0.29800478175877443, 1.171421050136773],
 [2.748488712473592, 3.4332645052174335],
 [0.6636626332674809, 0.3296510018543091],
 [2.662646134323486, 2.6565737073833513],
 [3.1811128923674663, 3.472217389920086],
 [3.084552650489785, 2.2588590093710414],
 [3.4828590606791443, 2.72392108379944],
 [-1.4199638973962447, 0.8090135480358618],
 [4.070403400940502, 2.951291099693843],
 [2.484825143388102, 3.214340289156354],
 [1.365045411346955, -0.13719428441342024],
 [0.9083663511854708, 0.06440431461485803],
 [3.3991448993792526, 2.9022533287506853],
 [3.9244685794964997, 3.3506981987178612],
 [0.5350374106461596, 1.2232277820718163],
 [3.2191138546433495, 3.0065743562599794],
 [0.018641103197320853, 0.29235588609723034],
 [0.5261286437821524, 1.0388170048470426],
 [1.2073509209922968, -0.3324284770074008],
 [2.8579969882613194, 4.038277778406835],
 [3.5232428904813315, 3.5318487730014034],
 [0.007720882330326118, -0.6168808229242169],
 [1.0492679979161201, 0.9369998065933355],
 [-1.5328847987431324, -0.23163378416985575],
 [-0.41949519165909327, -0.6393191132476826],
 [-0.9121911111462931, -0.2898762376496021],
 [-0.013736327194006805, 1.0452662735278058],
 [2.1018522140359996, 2.5930211657470967],
 [-0.032218148123971196, 1.230085296814402],
 [0.20352717800737335, -0.08523440865505],
 [3.7634118585967204, 2.741114287029227],
 [3.2723575545688224, 3.258618822813727],
 [0.28793011460672624, -0.81099071790311],
 [1.148317086648617, -2.7071226878366286],
 [-1.742919695935673, 1.4527644242334572],
 [2.647451909904468, 3.4879618815038262],
 [0.5089040883264622, -1.4407481985184196],
 [1.7302528211191097, 2.8030495251443632],
 [3.6295553666618554, 2.8517332534544213],
 [3.1792195046788514, 3.8291130211619704],
 [3.0828611715767344, 3.4259794150090124],
 [-0.5829486305975189, -1.1740296634432654],
 [-0.5219120916207514, 1.2539658479613942],
 [3.0062310212891425, 2.141639437316271],
 [3.53138785281316, 2.9432566275647956],
 [3.77740422395966, 3.0008305957471584],
 [-0.7118381264595716, -0.2809788728279539],
 [3.226398881210119, 2.9998297920579553],
 [3.2843762681735824, 2.515328674738321],
 [-0.6295719656120842, -0.19481797959297228],
 [2.906928847408494, 2.106640736398842],
 [-0.8843710775846532, -1.452657080357276],
 [-0.5262862977208965, 1.2722214188040908],
 [-0.872973271281121, 0.8144438039627917],
 [-0.21262999551087727, 0.8431174947950747],
 [2.269626246235757, 2.375254602640526],
 [3.9593668225544025, 2.928856535316963],
 [3.6163029515353005, 2.5661986381029926],
 [2.787969899556991, 3.143933084216097],
 [3.4184488113036076, 2.8647128442605703],
 [0.9448472844669686, -1.1187776212668132],
 [3.608840106846033, 3.328271407980294],
 [-1.5714337370174412, 0.2942650947847242],
 [-0.359974846955705, -1.1093571467890213],
 [3.1834022804051703, 2.167140527037704],
 [3.174519094850018, 3.6308805951388963],
 [2.189869951909971, 2.1812236824021376],
 [2.8528881632548067, 3.194071895498658],
 [-0.39534499277894886, -0.6481161616150263],
 [-0.02234780078071878, -0.007210160197046244],
 [1.0246508978539488, 1.337592840913042],
 [3.384942729328162, 3.029332672255315],
 [2.7992487087601186, 4.189710339025025],
 [-0.36632986245493065, 1.4193242999718274],
 [3.3210765434654035, 3.1482482991535896],
 [3.129266684573074, 2.8327208159166486],
 [0.3614195569132613, 1.5091040527329016],
 [2.3198397626424754, 3.007668442066844],
 [-1.1374550670810737, -0.44980720870673546],
 [-1.6680599400722917, -0.7990353877342014],
 [3.8281250544012284, 2.3802866194798935],
 [2.967795315196601, 2.797734991332108],
 [3.4948459368410814, 2.4049398463013825],
 [2.47720101783979, 3.4876173631802057],
 [3.054685162403755, 2.1795580364412093],
 [3.2494648015045637, 2.616718366371547],
 [0.8635867369444588, 1.3715988471024516],
 [0.4469402277187868, 1.7631545328982916],
 [-0.2213104514742908, 0.04297342350609582],
 [0.7507004311493115, 1.0005061436514904],
 [3.9732047307033342, 2.6154495274074874],
 [2.7592622813966843, 3.1551759834201145],
 [3.2104269753346206, 2.7579830208139517],
 [2.6846909511487747, 2.889016664172948],
 [3.382719099941961, 1.7736089650674083],
 [-0.1549589159071848, -0.9924992773262811],
 [3.2750180909640814, 2.9258694124985816],
 [3.3840864524924266, 2.8259147617369877],
 [-0.47859678209162904, -0.6973648355902023],
 [3.099062642479077, 3.1978255633341357],
 [2.7816230619365454, 3.666982828106018],
 [-0.5290710909316473, 1.3878644567547187],
 [-1.010749296916304, 0.5196418632132379],
 [3.738249406256451, 2.800988872095372],
 [2.3710611219902056, 2.662653450213273],
 [-0.05766346001236616, 0.1394860683187356],
 [3.2833058380668416, 3.58219709736091],
 [-1.9264043316393091, -0.4335516572240134],
 [1.4414070242190524, 0.7836113740546315],
 [2.601881322952827, 3.4637121673407334],
 [2.7937414987773224, 2.3589632394371165],
 [0.8287096058113333, -1.1169452111790477],
 [0.049408050245839136, 0.015072442499778257],
 [2.4702906124493804, 2.671382355470671],
 [0.5172704659013806, -0.7197566981841561],
 [-0.5634381178385752, -0.05334981629175853],
 [3.572329003961439, 2.919909712930682],
 [0.1797918082228199, -0.8553849327021266],
 [2.146364173749177, 3.0653681364169656],
 [-0.4435496276391044, 0.03458466634681497],
 [-0.26556951210431906, -1.006094498199089],
 [0.8020423175815613, -0.01921824630957957],
 [2.380473021166143, 3.3008307162175496],
 [-0.8355920666361928, 0.5143559406050374],
 [0.5605307501181245, -0.7474894739163606],
 [2.265048110035512, 2.6416316459967892],
 [4.086066718107087, 3.094199258868019],
 [2.8960201336632445, 2.9540089692859137],
 [0.33196765613886425, -0.6733364410867867],
 [2.6792412862082036, 3.4949628106780146],
 [0.4933399264058239, -0.6266347661174102],
 [2.9116642987281804, 2.77505785562916],
 [3.350964275401696, 2.3430778378108914],
 [2.6603579248872675, 2.786210894663075],
 [2.153470988545107, 3.436316039553232],
 [2.8642813358694976, 3.033413970157992],
 [0.9625784051691215, -0.9591866390414396],
 [3.4021125018305787, 3.011524299444485],
 [2.077197681501632, 2.1154757576687664],
 [-0.1006251977981523, -1.0366412380253922],
 [2.875436684358578, 2.72262179752859],
 [-0.8798113995562207, 0.9719543973391663],
 [3.487467380719034, 3.7106057706875024],
 [1.2853074427871567, 0.14112026171691483],
 [2.7464959746205966, 3.581218020853203],
 [2.824443872463048, 3.1331795416551604],
 [3.6060104805825706, 3.2045423428337316],
 [2.4974799969425807, 3.0865949850327117],
 [-0.6754227823478539, 1.6697447652898976],
 [-0.8929313945251028, 0.3050084477305372],
 [3.469096891373806, 3.1360053354017396],
 [2.8627158116985223, 2.990407855328947],
 [-1.1223819760494482, -1.1746658207638632],
 [-0.9178372005881059, -0.9846753502839615],
 [2.720369953177432, 3.228814633722775],
 [3.4290756477580207, 2.8309251455587034],
 [1.9336210704467276, 3.0744476924570803],
 [3.8870957137318927, 3.03332746406367],
 [0.5804689396718058, 1.3038043250410654],
 [2.163332581202971, 3.524376597207132],
 [3.0876348064980066, 3.6875065911616063],
 [-0.24823241141742505, 0.3996935580454786],
 [-1.0561443694190555, -0.3994253008372458],
 [2.9797568466332, 3.257678327203062],
 [3.655704219822241, 1.9627348929993789],
 [2.1267553461775544, 3.098995406168996],
 [1.8204387966613598, 2.054013941783896],
 [1.0104913886811713, 0.9738483677812636],
 [-2.049600572289606, -0.49925424458464396],
 [2.5625153375046383, 2.7285532495348397],
 [3.3957881027684613, 2.502793986006826],
 [1.1933554048557433, -0.26267839246442365],
 [-0.0663436275722224, 1.365928838414808],
 [-1.4927346627633222, 0.3420765184349746],
 [-0.7032440880822468, 1.554755515936172],
 [3.4415496731202526, 2.465164911596797],
 [2.7068489834866174, 3.3236293935756347],
 [3.9614602886168426, 2.8968517472975543],
 [0.89753506945278, 1.0591146399348845],
 [3.4032416106986663, 2.0771968255109687],
 [0.8412311321412008, -0.26142598280009827],
 [-0.8010989974282529, -1.04618637032617],
 [-1.3605612688138882, -0.22187671774642226],
 [-0.9457583038511481, -0.9142033351810103],
 [0.30873891313209667, 0.8816476902449996],
 [1.4708476983861134, -0.6333741614323367],
 [3.4105978568765076, 2.3270962938359614],
 [0.6807762584926872, 0.3190508981521644],
 [0.8317717129897895, 1.5190340113233072],
 [-0.6847013495756882, -0.07047586116553842],
 [-0.5926263895126221, 0.7369399283269445],
 [0.28599455004630286, -0.9980214501293055],
 [-0.44450247072101495, 0.6439779068059945],
 [0.953809305540609, 0.27353609988187216],
 [0.6678996845708064, 1.5444455050849213],
 [3.333892024696918, 2.7685570455339423],
 [-1.1735331718210156, -1.0386705291828529],
 [-0.10445457025516264, -0.992198207432028],
 [-2.3812198785722596, 0.5774782666960828],
 [0.029362030681380928, 1.574942845907442],
 [-0.307951475285576, 0.41180807819795967],
 [2.852435852758025, 3.137375222803647],
 [-1.3350529228185792, 0.6353316878308268],
 [3.422626900702981, 3.723997618236947],
 [1.0878847906167113, 1.8328581039554381],
 [-0.12685724460588813, -1.0945508472658991],
 [2.182718269512383, 2.958602583148231],
 [2.914286897735059, 3.713458362054754],
 [-0.2008749897883501, -0.4574099212645889],
 [2.995627237708591, 2.7581088666811184],
 [3.672392175566532, 3.0378325549553233],
 [2.736316751881088, 3.691636240370996],
 [3.4651122828210683, 3.6301761547406075],
 [0.23478151149191112, 0.48782245550283204],
 [-0.4159593461007176, 0.6856734499864355],
 [3.0207092458807536, 3.38569825213821],
 [2.99456812639767, 3.134033282493832],
 [2.272359028833752, 2.163966254706684],
 [3.3263338700588223, 3.127618102604538],
 [1.1832306862354176, -0.5860668564322333],
 [1.8849016124581357, -0.3512225734480826],
 [2.3901905366428773, 1.7978653944303233],
 [3.5186302921693318, 3.415233288520473],
 [3.1156526028270197, 3.6194101764292785],
 [-0.2778431574725113, 0.9026940526208068],
 [2.8734933645244887, 3.9439267820774617],
 [0.8354399210931627, 0.24363424568344502],
 [0.8765362174725616, 1.8542440004122156],
 [0.9998114056159516, 0.1299098717899012],
 [2.755205617010477, 3.92018409124765],
 [-0.26765097779735325, -0.22721022904494725],
 [2.514777809977817, 3.4382264735749444],
 [2.657862963139512, 3.3728471197741006],
 [3.576951187399446, 3.1227536389640873],
 [3.099754001797307, 2.0899895727717785],
 [3.1793833172413732, 2.8532057053682927],
 [-0.059991401825511596, 0.09525251173556964],
 [2.778244365567263, 2.7055675195710993],
 [0.35919437483424826, 1.593489751227171],
 [3.8571065566701472, 3.2096035615277634],
 [-0.07312192616221837, -0.7729827525852702],
 [-0.03951811053166313, -0.8824734256000918],
 [-0.666239477013906, -1.2398707248535716],
 [2.4854311905823723, 3.131619941151607],
 [2.610793897561527, 2.766493208833976],
 [3.048113056742991, 3.596104179625185],
 [-0.8589433770388482, -0.4698659152167409],
 [2.955161555939326, 3.3093971641501225],
 [-0.4564753196929517, 0.1960254625937407],
 [3.5210071563081327, 3.886872335704798],
 [-1.6494587171473534, 0.8576711251157653],
 [1.2603053324126574, -0.9231746114376692],
 [2.3895840763623166, 3.674284812350698],
 [-0.49029657082401595, 1.0647864948225958],
 [2.714668214366406, 2.7927452055625124],
 [1.1368056222414653, -0.5486412615015356],
 [0.3940043551780911, -0.4203989331300182],
 [1.063187237706002, 0.2312088526543401],
 [3.6467062323830386, 2.5340916952420116],
 [0.17059989310635232, 0.6249010293629813],
 [-0.06504126890092723, -0.9235989941822599],
 [3.597110568380038, 3.711078309400734],
 [2.2526689004782354, 2.8472644283576796],
 [-0.774133139222836, 1.5842600562109153],
 [2.9728938685135, 2.198901172172951],
 [-1.3806068250396146, 1.0867721674972608],
 [-0.530232544758548, -0.24108103368838626],
 [-0.3004901496194447, 0.6243298269693522],
 [2.4910810527120737, 2.4213982708258563],
 [3.3285843977248577, 2.9988746974399856],
 [0.14107086398678953, 1.4069123300751674],
 [2.726468290579359, 2.845125217031872],
 [2.9606584108375285, 3.380396202520731],
 [-0.29850726795229415, -0.6711836555086634],
 [2.7868797438469786, 3.5520508736943746],
 [4.031709865743338, 3.1983295228413917],
 [2.4385997089657234, 3.567802129276842],
 [1.3175565806300806, -0.225485676537257],
 [-1.3344029105108186, -1.5331799210630725],
 [0.7038524098624278, -0.5064787569139104],
 [0.827959897084177, 1.1946579022421209],
 [2.523805454552545, 3.3103849185303393],
 [0.5462484989805613, 0.1407834011955556],
 [-0.16447224932209045, -1.0622847524177794],
 [2.656186737704345, 3.1969992138227505],
 [-0.45627905672175445, -0.42927915743265715],
 [-0.8956226896840942, -1.6167241637593393],
 [1.2227273782956605, 0.7267777174402084],
 [2.874435298443434, 4.238385822981458],
 [-1.365303822511754, -1.0667319571330476],
 [3.7312218191477755, 3.227029536009529],
 [1.9876413392000056, 3.424275642173552],
 [2.5847303532086205, 3.6087421354987104],
 [-1.0159305117385846, -0.28730188900517706],
 [-1.2463799455403257, -0.9572979001429889],
 [3.5945064113843523, 3.173149117138588],
 [3.0226388338956305, 3.283835379248959],
 [0.7343034116433089, 0.21437302481820808],
 [0.25401460520521274, 0.7425143695201689],
 [2.7010471191816907, 2.9632115063260973],
 [-0.6125942531883626, 0.5089247063256368],
 [2.366710177758242, 3.1235984602143114],
 [1.3333190523105658, -0.3931367383921331],
 [-0.7139050161302947, -0.34802325117750116],
 [2.964142058302058, 3.2479137555107975],
 [0.21276201703317182, 0.05739956239323806],
 [3.2650301726050137, 2.9201766797876747],
 [0.08825118920410505, 0.5269412540930901],
 [1.3932605173123906, -0.08376976385830917],
 [3.6219413871706676, 3.5889521359240844],
 [3.5938031775219352, 3.1162707633497777],
 [3.251769304511085, 3.7901349463207397],
 [3.351509821571243, 2.094349208039664],
 [0.46738477393952166, -0.3740238634075434],
 [3.3978517397108514, 2.293394458785704],
 [3.692766000959439, 3.1426338616841623],
 [3.2176819835466333, 3.1339726340641474],
 [3.232967169808437, 2.800763595973901],
 [2.1680223839362247, 3.0988350223768726],
 [0.8911077735256963, 0.2840338914625887],
 [1.861357802418688, 3.69789047817443],
 [1.2924303912574984, 3.133323611342584],
 [2.8268661788915956, 2.392707756320652],
 [3.4891225124032625, 3.3753844052508546],
 [-1.2731563285839094, 1.207492078041777],
 [3.352111372606209, 3.7600757086314536],
 [3.66800530416072, 2.905972773083453],
 [0.7537595947632416, -1.3217334572497295],
 [1.6078354584569996, -2.444597668761012],
 [-0.287794155459982, 0.790491682646018],
 [3.601050735780591, 2.086629991098822],
 [3.8251101815916635, 2.557396039284797],
 [-2.1806478202371977, 0.5266394878972454],
 [-0.3248071645528655, 1.0697355947351743],
 [-0.4541141833984531, -0.715508060094054],
 [-0.08609965475487315, 0.16304830620545813],
 [0.1117235929777853, 0.06497987337686585],
 [3.198697336587607, 3.3275981951303066],
 [2.6021176924913347, 2.1744713588760893],
 [2.9458521459867693, 3.740647410315484],
 [-0.3899002169909129, 0.41003256581049796],
 [-0.488625070383078, 0.08054584523057426],
 [3.5342721213452717, 2.749327875853442],
 [3.3280520027484335, 3.3257442939280533],
 [3.1589507908206973, 2.8100489279760237],
 [-1.5668169224968258, 0.3110059378751947],
 [2.671065069605009, 2.925551358125089],
 [0.7317306402363108, -0.20325724147161903],
 [1.144844145803122, 0.34548268561115303],
 [1.0444994866936064, 0.4976623044651587],
 [-0.3801719903119817, -0.6994287804338023],
 [-0.32998705036581305, 0.9356430203365887],
 [3.114839994597297, 3.9733621485276562],
 [-1.718287061194924, -0.13553831402630867],
 [2.4936999754902973, 3.370742289928148],
 [3.208307340174483, 4.390475254707666],
 [2.8728428302999176, 2.9802244163491154],
 [2.881062844426585, 3.6482522522144083],
 [-0.14221708452759615, -0.17439349340233581],
 [0.14810845663589606, 1.1360776538080288],
 [3.02845352706938, 2.8420613688292495],
 [-0.37612554666458986, -0.4008789688250792],
 [-0.47106815929710416, -1.1641285164157111],
 [3.467674638563639, 3.210649419878074],
 [0.3376028288014988, -1.3956183272724214],
 [0.9344997950434469, -0.7501420977536598],
 [1.5437686244529487, 0.27549689549371903],
 [-0.5427765562654832, -0.07828253926783661],
 [0.33470171521490866, -0.5954627037654896],
 [3.209122238078535, 2.292306834018612],
 [0.43108807431421814, 0.07219428748033453],
 [2.6733788602870407, 2.907738170951684],
 [1.203418192225393, 1.3012333821132602],
 [0.7309057907253983, -1.2971934677722818],
 [2.9340866182879717, 3.2242357403237323],
 [2.3735455062829045, 2.606070670057897],
 [3.249576354938417, 3.601090251473594],
 [2.221516059496124, 2.2691892876160895],
 [2.8348077347175917, 2.2745779636879138],
 [2.9306104726238287, 3.080439903150149],
 [1.5456456245530155, -0.22371091244564903],
 [-0.6004372167787508, -1.0688572179839104],
 [-0.7432596311808121, -0.9614223539522136],
 [3.6171587136409693, 3.304181638939645],
 [0.14789296383158865, -1.4541243426274706],
 [0.2201059228627741, 0.4110105829070647],
 [3.11817329247578, 2.5277131774160875],
 [3.146195210129167, 3.1033650020773926],
 [3.179229965186934, 4.24985974633187],
 [2.5584099134733904, 2.884876085782111],
 [1.1974188669926902, -0.19888966691025894],
 [2.319372164532888, 3.05076319804842],
 [-0.5394520766670793, -1.541054501782372],
 [3.0121381666678033, 3.677015674678318],
 [0.9121441785274069, 1.1942267253579697],
 [3.0414916055523595, 3.6248576229532654],
 [-0.41257002087346745, 1.3648047537509091],
 [3.8029858140071955, 2.478584301718973],
 [2.525934268210875, 1.8789365160609262],
 [0.07912444369781685, 0.7993841725126744],
 [2.4203373278799662, 2.7998745625821124],
 [-0.3354465530803439, 0.5803307534001644],
 [2.6967081070420673, 2.945138570160165],
 [2.5968589428721365, 2.661447614481798],
 [3.3148922573209116, 3.110501820797051],
 [-0.3648687401083515, -0.28673547185350257],
 [-0.3661849217833602, -0.6934655242480043],
 [2.867716056295292, 3.4885908821626224],
 [2.7961911018688377, 2.2750458384794197],
 [2.1644164145674383, 3.218556478846738],
 [0.4185787437083088, -0.6325458150315751],
 [3.5158536742018116, 2.95481166895466],
 [2.631513080287978, 2.6906908700836416],
 [2.67724579950399, 3.5569595338271305],
 [0.1162059806060796, 0.762650377889224],
 [1.0107115047538315, -0.09076878799055173],
 [1.3911278189425977, -0.13663198983685154],
 [-0.26700933197604004, 0.9766032139091303],
 [-0.8445495837020328, -0.03199244028600801],
 [3.1857324972632575, 3.9161533983438352],
 [1.588390467876622, 0.1454191351193014],
 [0.9224171557708682, -0.6378599846211848],
 [-0.855863068662826, -0.2840632273872881],
 [1.9793759899929813, -0.8265934002023423],
 [0.9440508903438, 0.5575828523935861],
 [-1.2768300164569728, 1.4237248855679452],
 [-2.288237926248231, -1.7132486864358087],
 [-0.5921911320777613, -1.0303098264335475],
 [3.259530527592155, 2.591663019143951],
 [3.0107396271648326, 2.511714006907719],
 [2.751952552022104, 3.4012869575863487],
 [3.6432385063164885, 3.138501655319918],
 [3.4078057803044937, 4.041077478987494],
 [2.1178888446658926, 2.9728039093011085],
 [3.6798157764135833, 3.282661519638706],
 [3.36438360006556, 3.1058382378983826],
 [3.1040790758023404, 2.9680117952194025],
 [3.3800897496706153, 2.202462972210541],
 [-0.07004855105588846, -1.4495994206316984],
 [2.8945006857911535, 3.4632810352915224],
 [2.9221302981869792, 1.5062749067004813],
 [3.089134507971404, 2.2382595258252045],
 [2.4649070774532085, 4.07490100410741],
 [3.2426597846113214, 2.378472620498816],
 [2.228011013160181, 3.4498209384432656],
 [2.5911719165459264, 3.0732807290442308],
 [3.447963273603751, 3.3975464724987656],
 [3.404723085671628, 3.2552443907216113],
 [1.7932109158648604, 0.4107722602251995],
 [-1.234282985148641, 1.9180278076338928],
 [0.8048900583374947, 1.559914502657956],
 [3.0652230613895144, 2.5607188768432945],
 [3.350472811391246, 3.1340056472739364],
 [2.810228018105401, 2.8362520133046165],
 [3.1487374867780376, 3.113568872860009],
 [2.4324949940710443, 3.076175073883033],
 [-0.6441342200201419, 1.1523835988338538],
 [1.1405525015879128, -0.3090369010331575],
 [1.1916372186576063, 1.6060106949881845],
 [3.033174909588307, 3.2187088008215894],
 [0.7319430434155801, 0.3915083394951812],
 [2.185717526949622, 2.5090983971633745],
 [0.9386252266608126, -0.6772461090486613],
 [-0.22706325835834335, -0.5191854955276989],
 [-0.453692981497249, -0.47852180768462677],
 [3.0335593316882474, 2.742466803960998],
 [1.8579188075587691, 3.1095154333597677],
 [1.3464109435404168, 0.9949008377158691],
 [2.24387567274501, -1.971904359692604],
 [0.9906169462818061, -0.1893243470131887],
 [2.3081524711889423, 2.2028863846868765],
 [3.1762862961293283, 3.1731724335335896],
 [2.9336049770999244, 3.299776868096602],
 [2.433930653861836, 2.1830154610506316],
 [-0.13326620663113783, -0.37146797920961966],
 [2.5267343975665573, 2.973709517530221],
 [0.4075405723168199, -1.4102698246183245],
 [2.6392733996019886, 3.184262178505766],
 [3.993589183095866, 3.2816040618389968],
 [-1.3597127046353616, -1.2895273432501333],
 [2.300982909791787, 3.5345833502596946],
 [-0.8044294987268793, -1.6364128983877968],
 [0.8465282844235061, -0.47958595326783576],
 [2.912594837413425, 3.001909667471459],
 [2.6094017119018784, 2.4029230129397385],
 [2.9264672097772517, 2.578605986573066],
 [3.6644196133610523, 2.3724621550597296],
 [0.14850485362167035, 0.11980813677579531],
 [2.3911227730277504, 3.985682086534092],
 [1.7494511286381909, 3.395839769435011],
 [1.379088111049455, 0.7597430569733866],
 [3.191972834972405, 3.06292741459173],
 [1.2794348439537664, -0.18820105796984918],
 [-0.9864478863242372, 1.0690132406860626],
 [3.972738594013023, 2.60787944676075],
 [0.352866591224531, -0.11644150824300875],
 [0.9691835686887775, -0.9954913496100612],
 [3.366547814523951, 2.7587333270241876],
 [-0.6087314600283987, -0.16297936208528138],
 [-0.7900133675571189, -0.7662059527879855],
 [-1.4913781331347458, -0.3062667829579504],
 [1.0644140043762134, -0.4316688802452635],
 [2.881333685585583, 4.231816330784231],
 [3.9072383987789303, 2.2895673073986123],
 [3.063188434969431, 3.4160310031993166],
 [3.1894964979796536, 2.7518003401073106],
 [0.05061497746459626, -0.11991248281206213],
 [0.403510611916212, 0.08169546384238874],
 [-1.3574542784806898, -1.357537826753475],
 [4.009250957856397, 2.703764068780419],
 [0.2248410881251251, 0.20976489773086782],
 [0.007360591371091639, 1.2472133917688852],
 [-1.222524840070517, 1.251712540313258],
 [-0.7064586253157181, -0.14490206028258618],
 [1.2233804862236504, 0.003818870640195979],
 [0.5548076219860761, 1.1953810135668363],
 [3.498125873266776, 2.913074512660586],
 [3.4644992258632774, 3.8352776795948476],
 [3.281188883601909, 3.4642703251303177],
 [2.3695458656154655, -0.4272405321386757],
 [0.05337680512349036, -1.5222690212888015],
 [3.022715137702981, 3.1794713369283434],
 [-1.1887490538453531, 0.23152255824417411],
 [3.0671150780580336, 1.8493925743163953],
 [-1.097482973252559, 0.44377535909455834],
 [2.9503213947928018, 3.1652054069922824],
 [-0.0037571540662578863, -0.2237790805417408],
 [-0.736030993551402, -1.314814863729379],
 [-1.1933905774875997, 1.6964945034956962],
 [-1.245758568973453, 0.7876845584956895],
 [-0.24461597279476974, 1.106447333879627],
 [3.0908425135671633, 2.849800081184169],
 [4.17148448992426, 2.7066204194703003],
 [0.9159869160980336, 0.7711238004788284],
 [2.705837863353758, 2.8132978032462823],
 [1.286559450763713, -0.616078790993532],
 [3.526832446803886, 2.083411203792124],
 [0.04211727923127136, 0.3592373944264551],
 [1.6427623828229712, 0.014400190130595044],
 [3.6100333141538448, 3.7229032743946777],
 [0.2055222686535367, -2.7438807005703585],
 [-2.1655320345981393, -1.5952896012755928],
 [3.0413523105260434, 2.772362134934384],
 [-0.7692592054803452, -1.2599909945311893],
 [-0.5722525968731, -1.0957417880243792],
 [0.4161779196176454, -0.9384027032744411],
 [0.5510494196473434, -0.23021171293320025],
 [1.36523637268077, 0.48526708943519437],
 [1.527473737029303, 1.7586420459959393],
 [3.385229465860831, 2.5275458810806533],
 [2.582393237554753, 3.1098401867247545],
 [2.764271477628532, 1.726212736509441],
 [2.9226151725919176, 2.984092211653762],
 [3.382283005222344, 2.375096557967333],
 [1.0963618136349345, 0.16795530624634367],
 [0.5018244945038683, -0.49459300203297446],
 [3.2508377102514823, 2.8702294933485244],
 [-1.2534684828542926, 0.22898538753868958],
 [0.593103407370136, -0.3017834185285886],
 [2.430385370020505, 4.273458422556113],
 [3.1615516509683252, 3.5119520780702143],
 [-0.3342056165078333, 0.7158752533995206],
 [3.2084181438818775, 3.3135989575028373],
 [-0.12053999094210403, -1.0916307680755466],
 [0.5283630931865074, -0.3423784904630562],
 [2.5475022695844896, 2.2213382791940495],
 [2.280828834664366, 2.841947421780328],
 [2.0992055008821495, 3.813296644282992],
 [0.49112234767932045, -1.0883490554742419],
 [2.889347040459979, 3.20745013491598],
 [0.8844882335291433, -0.14306310680830464],
 [3.13737683938748, 3.1875362725691625],
 [-0.7404245027318411, 0.5627245313056246],
 [2.5659537040555, 2.5529660806124044],
 [2.9059521347895836, 3.09788703996329],
 [0.2911789304753991, 0.42006483702190417],
 [2.1525965119720576, -0.7601613182393744],
 [0.35009384611840094, 2.310447479429575],
 [3.4088435370374386, 3.1438244458947833],
 [2.7137527909669363, 3.0368584197657262],
 [-0.47738614550386904, 0.07649289265175319],
 [0.6413826143005864, -0.535880841649232],
 [2.495280044155492, 3.344973157043693],
 [2.7562470193950888, 3.063904987254759],
 [2.3298091669069287, 2.0013376646965457],
 [-0.27263762779854533, -0.9006164761612259],
 [2.707633804953344, 3.0784806197503327],
 [2.8959562338250042, 3.099714033031465],
 [1.7669183825593686, -0.2349842513377137],
 [3.702187361147054, 2.636121853342347],
 [2.62377875640534, 2.8652976985597394],
 [3.014473198057002, 3.471134731455271],
 [0.028461410907849906, -1.455760583168695],
 [3.1319163311114724, 2.6590941479936596],
 [2.461329816234051, 3.3565407334882873],
 [3.437213989665808, 4.340031992799637],
 [2.7960095140742958, 2.642405552826782],
 [-0.6797231884876173, -0.68256779458632],
 [-0.5863943035475916, -0.3446395865770017],
 [2.5302170512034867, 3.1576843154655285],
 [3.5839142121259098, 3.5959181902735575],
 [0.47201806732351576, -0.17797516664651486],
 [-0.3091581095660412, -1.0220054043625972],
 ...]

In [3]:

%matplotlib nbagg
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

df = pd.DataFrame({"x": [v[0] for v in vector_set],
                  "y": [v[1] for v in vector_set]})

sns.lmplot("x","y", data=df, fit_reg=False, size=6)
plt.show()

K평균 알고리즘 구현 : 4개 군집으로 군집화

1. 텐서 데이터 생성

In [4]:

import tensorflow as tf

vectors = tf.constant(vector_set)  # 생성한 난수를 상수텐서로 바꿈
k = 4                             # 군집개수 지정 
vectors.get_shape()              # 텐서의 구조 확인

/usr/local/lib/python2.7/dist-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
  "This module will be removed in 0.20.", DeprecationWarning)

Out[4]:

TensorShape([Dimension(2000), Dimension(2)])

2. 중심 값 생성

In [5]:

centroides = tf.Variable(tf.slice(tf.random_shuffle(vectors),[0,0],[k,-1])) # 랜덤으로 데이터에서 k개 데이터 선택
centroides.get_shape()

Out[5]:

TensorShape([Dimension(4), Dimension(2)])

3. 중심과의 거리 계산

• 중심은 4개, 전체 값은 2000개 (차원의 크기가 다름)
• tf.expand함수로 차원을 추가하여 3차원으로 만들어 뺄샘을 할수 있게 크기를 맞춤

In [6]:

expanded_vectors = tf.expand_dims(vectors,0)
expanded_vectors.get_shape() 

Out[6]:

TensorShape([Dimension(1), Dimension(2000), Dimension(2)])

In [7]:

expanded_centroides = tf.expand_dims(centroides,1)
expanded_centroides.get_shape()

Out[7]:

TensorShape([Dimension(4), Dimension(1), Dimension(2)])

• 브로드캐스팅기능으로 tf.sub함수는 두텐서의 각원소를 어떻게 빼야할지 스스로 알아낼수 있음
• 크기가 1인 차원은 텐서 연산 시 다른 텐서의 해당 차원 크기에 맞게 계산을 반복하므로 마치 차원이 늘어난 것과 같은 효과를 가짐
• d2차원(크기가2인)에서 뺄샘이 이루어짐
• d0차원은 각각 크기가 1,4로 다름 그럼 벡차의 차원을 4로 늘려서 연산함 / d1차원도 동일함

3.1 유클리드 제곱거리를 사용하는 할당단계의 알고리즘

In [8]:

diff = tf.sub(expanded_vectors,expanded_centroides)  # 중심 - 각x,y값을 뺀것
sqr = tf.square(diff) # diff 값을 제곱
distance = tf.reduce_sum(sqr,2) # 제곱한 값의 합
assignments = tf.argmin(distance,0) #거리의 합이 가장 작은 값의 인덱스(0차원)

• 각 차원의 크기를 확인해보면 요래요래

In [9]:

diff.get_shape()

Out[9]:

TensorShape([Dimension(4), Dimension(2000), Dimension(2)])

In [10]:

sqr.get_shape()

Out[10]:

TensorShape([Dimension(4), Dimension(2000), Dimension(2)])

In [11]:

distance.get_shape()

Out[11]:

TensorShape([Dimension(4), Dimension(2000)])

In [12]:

assignments.get_shape()

Out[12]:

TensorShape([Dimension(2000)])

In [13]:

#한줄로 하면 이렇게 
assignments = tf.argmin(tf.reduce_sum(tf.square(tf.sub(expanded_vectors,expanded_centroides)),2),0)

#두줄 정도가 보기 좋으니 이렇게 
distance = tf.reduce_sum(tf.square(tf.sub(expanded_vectors,expanded_centroides)),2)
assignments = tf.argmin(distance,0)

3.2 새로운 중심 계산

• 마지막 수정단계로 새롭게 군집을 만들어 새로운 중심을 다시 계산하는 단계
• k개 군집에 속하는 점들의 평균을 가진 K개의 텐서를 합쳐서 means 텐서를 생성

In [14]:

means = tf.concat(0, [tf.reduce_mean(tf.gather(vectors, 
                                               tf.reshape(tf.where(tf.equal(assignments,c)),[1,-1])),
                                     reduction_indices=[1]) for c in xrange(k)])

• 1) equal 함수로 한 군집에 매칭되는 assignments텐서의 각 원소위치를 True표시하는 불리언텐서 생성(dimension 2000)
• 2) where 함수로 불리언텐서에서 true로 표시된 위치값을 갖는 텐서 생성 (dimension 2000 * dimension1)
• 3) reshape 함수로 c군집에 속한 vectors텐서의 포인트들의 인텍스로 구성된 텐서 생성 (dimension1 * dimension2000)
• 4) gather 함수로 c군집을 이루는 점의 좌표를 모은 텐서 생성 (dimension1 dimension2000 dimenrion2)
• 5) reduce_meand 함수로 c군집에 속한 모든 점의 평균 값을 가진 텐서 생성(dimension1 * dimension2000)
• (http://goo.gl/5s0Y2S)에서 각 함수에 대한 자세한정보 얻을수 있음

함수	설명
tf.reduce_sum	지정한차원을 따라 원소들을 더함
tf.reduce_prod	지정한 차원을 따라 원소들을 곱함
tf.reduce_min	지정한차원을 따라 최소값을 계산
tf.reduce_max	지정한차원을 따라 최대값을 계산
tf.reduce_mean	지정한차원을 따라 평균을 계산
tf.argmin	지정한 차원을 따라 가장 작은 값의 원소가 있는 인텍스를 리턴
tf.argmax	지정한 차원을 따라 가장큰 값의 원소가 있는 인덱스를 리턴

3.3 그래프 실행

• 루프를 구성하고 중심을 means 텐서의 새값으로 업데이트함
• 1) means 텐서의 값을 centroids에 할당 하는 연산을 작성해야함

In [15]:

update_centroides = tf.assign(centroides, means)
init_op = tf.initialize_all_variables()

In [16]:

sess = tf.Session()
sess.run(init_op)

for step in xrange(100):
   _, centroid_values, assignment_values = sess.run([update_centroides,
                                                    centroides,
                                                    assignments])

In [17]:

print centroid_values

[[ 3.35460877  3.11880803]
 [-0.71416128  0.12714821]
 [ 2.53997254  2.81049085]
 [ 0.73785478 -0.22789095]]

In [18]:

data = {"x": [], "y": [], "cluster": []}

for i in xrange(len(assignment_values)):
  data["x"].append(vector_set[i][0])
  data["y"].append(vector_set[i][1])
  data["cluster"].append(assignment_values[i])

df = pd.DataFrame(data)
sns.lmplot("x", "y", data=df, 
           fit_reg=False, size=7, 
           hue="cluster", legend=False)
plt.show()

문서를 어떻게 벡터로 표현하나?(윈도우 기반 메트릭스)

In [22]:

%matplotlib nbagg
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

la = np.linalg
words = ["I", "like", "enjoy","deep", "learning", "NLP", "flying", "."]
X = np.array([[0,2,1,0,0,0,0,0],
              [2,0,0,1,0,1,0,0],
              [1,0,0,0,0,0,1,0],
              [0,1,0,0,1,0,0,0],
              [0,0,0,1,0,0,0,1],
              [0,1,0,0,0,0,0,1],
              [0,0,1,0,0,0,0,1],
              [0,0,0,0,1,1,1,0]])

U, s, Vh = la.svd(X, full_matrices=False)

for i in xrange(len(words)):
    plt.text(U[i,0], U[i,1], words[i])