Tf–idf 편집하기

{{소문자}}
==개요==
; tf–idf, TFIDF, term frequency–inverse document frequency
; 빈도-역빈도 법, 단어 빈도-역문서 빈도
* 정보 검색과 텍스트 마이닝에서 이용하는 가중치
* 문서내 단어의 가중치를 구하는 알고리즘
* 정보검색론의 기본 개념중 하나
* 문서 간의 유사도 측정
* 특정 단어의 문서내 중요도 측정
* 핵심어 추출
* 검색 랭킹 결정

==예시==
{| 
|

{| class="wikitable"
|+ Document 1
! term
! count
|-
| this || 1
|-
| is
| 1
|-
| a
| 2
|-
| sample
| 1
|-
| 계
| 5
|}

|

{| class="wikitable"
|+ Document 2
! term
! count
|-
| this || 1
|-
| is
| 1
|-
| another
| 2
|-
| example
| 3
|-
| 계
| 7
|}

|}

{| class='wikitable'
! 단어 "this"
|-
|
* <math>{\displaystyle \mathrm {tf} ({\mathsf {''this''}},d_{1})={\frac {1}{5}}=0.2}{\displaystyle \mathrm {tf} ({\mathsf {''this''}},d_{1})={\frac {1}{5}}=0.2}</math>
* <math>{\displaystyle \mathrm {tf} ({\mathsf {''this''}},d_{2})={\frac {1}{7}}\approx 0.14}{\displaystyle \mathrm {tf} ({\mathsf {''this''}},d_{2})={\frac {1}{7}}\approx 0.14}</math>
|-
|
* <math>{\displaystyle \mathrm {idf} ({\mathsf {''this''}},D)=\log \left({\frac {2}{2}}\right)=0}{\displaystyle \mathrm {idf} ({\mathsf {''this''}},D)=\log \left({\frac {2}{2}}\right)=0}</math>
|-
|
* <math>{\displaystyle \mathrm {tfidf} ({\mathsf {''this''}},d_{1},D)=0.2\times 0=0}{\displaystyle \mathrm {tfidf} ({\mathsf {''this''}},d_{1},D)=0.2\times 0=0}</math>
* <math>{\displaystyle \mathrm {tfidf} ({\mathsf {''this''}},d_{2},D)=0.14\times 0=0}{\displaystyle \mathrm {tfidf} ({\mathsf {''this''}},d_{2},D)=0.14\times 0=0}</math>
|}

{| class='wikitable'
! 단어 "example"
|-
|
* <math>{\displaystyle \mathrm {tf} ({\mathsf {''example''}},d_{1})={\frac {0}{5}}=0}{\displaystyle \mathrm {tf} ({\mathsf {''example''}},d_{1})={\frac {0}{5}}=0}</math>
* <math>{\displaystyle \mathrm {tf} ({\mathsf {''example''}},d_{2})={\frac {3}{7}}\approx 0.429}{\displaystyle \mathrm {tf} ({\mathsf {''example''}},d_{2})={\frac {3}{7}}\approx 0.429}</math>
|-
|
* <math>{\displaystyle \mathrm {idf} ({\mathsf {''example''}},D)=\log \left({\frac {2}{1}}\right)=0.301}{\displaystyle \mathrm {idf} ({\mathsf {''example''}},D)=\log \left({\frac {2}{1}}\right)=0.301}</math>
|-
|
* <math>{\displaystyle \mathrm {tfidf} ({\mathsf {''example''}},d_{1},D)=\mathrm {tf} ({\mathsf {''example''}},d_{1})\times \mathrm {idf} ({\mathsf {''example''}},D)=0\times 0.301=0}{\displaystyle \mathrm {tfidf} ({\mathsf {''example''}},d_{1},D)=\mathrm {tf} ({\mathsf {''example''}},d_{1})\times \mathrm {idf} ({\mathsf {''example''}},D)=0\times 0.301=0}</math>
* <math>{\displaystyle \mathrm {tfidf} ({\mathsf {''example''}},d_{2},D)=\mathrm {tf} ({\mathsf {''example''}},d_{2})\times \mathrm {idf} ({\mathsf {''example''}},D)=0.429\times 0.301\approx 0.129}{\displaystyle \mathrm {tfidf} ({\mathsf {''example''}},d_{2},D)=\mathrm {tf} ({\mathsf {''example''}},d_{2})\times \mathrm {idf} ({\mathsf {''example''}},D)=0.429\times 0.301\approx 0.129}</math>
|}

==같이 보기==
{{z컬럼3|
* [[명사구]]
* [[페이지랭크]]
* [[단어 임베딩]]
* [[단어수 세기]]
* [[Okapi BM25]]
* [[상호의존정보]]
* [[잠재 의미 분석]](LSA)
* [[벡터 공간 모델]]
* [[잠재 디리클레 할당]]
* [[쿨백-라이블러 발산]]
* [[콘텐츠 기반 필터링]]
}}

==참고==
* {{위키백과|tf-idf}}
* {{다음백과|tf-idf}}
* {{네이버백과|tf-idf}}

[[분류: 자연어 처리]]
[[분류: 알고리즘]]
@@ 63번째 줄: / 63번째 줄: @@
 |-
 |
-* <math>\mathrm {tf} ({\mathsf {''this''}},d_{1})={\frac {1}{5}}=0.2</math>
+* <math>{\displaystyle \mathrm {tf} ({\mathsf {''this''}},d_{1})={\frac {1}{5}}=0.2}{\displaystyle \mathrm {tf} ({\mathsf {''this''}},d_{1})={\frac {1}{5}}=0.2}</math>
-* <math>\mathrm {tf} ({\mathsf {''this''}},d_{2})={\frac {1}{7}}\approx 0.14</math>
+* <math>{\displaystyle \mathrm {tf} ({\mathsf {''this''}},d_{2})={\frac {1}{7}}\approx 0.14}{\displaystyle \mathrm {tf} ({\mathsf {''this''}},d_{2})={\frac {1}{7}}\approx 0.14}</math>
 |-
 |
-* <math>\mathrm {idf} ({\mathsf {''this''}},D)=\log \left({\frac {2}{2}}\right)=0</math>
+* <math>{\displaystyle \mathrm {idf} ({\mathsf {''this''}},D)=\log \left({\frac {2}{2}}\right)=0}{\displaystyle \mathrm {idf} ({\mathsf {''this''}},D)=\log \left({\frac {2}{2}}\right)=0}</math>
 |-
 |
-* <math>\mathrm {tfidf} ({\mathsf {''this''}},d_{1},D)=0.2\times 0=0</math>
+* <math>{\displaystyle \mathrm {tfidf} ({\mathsf {''this''}},d_{1},D)=0.2\times 0=0}{\displaystyle \mathrm {tfidf} ({\mathsf {''this''}},d_{1},D)=0.2\times 0=0}</math>
-* <math>\mathrm {tfidf} ({\mathsf {''this''}},d_{2},D)=0.14\times 0=0</math>
+* <math>{\displaystyle \mathrm {tfidf} ({\mathsf {''this''}},d_{2},D)=0.14\times 0=0}{\displaystyle \mathrm {tfidf} ({\mathsf {''this''}},d_{2},D)=0.14\times 0=0}</math>
 |}
@@ 78번째 줄: / 78번째 줄: @@
 |-
 |
-* <math>\mathrm {tf} ({\mathsf {''example''}},d_{1})={\frac {0}{5}}=0</math>
+* <math>{\displaystyle \mathrm {tf} ({\mathsf {''example''}},d_{1})={\frac {0}{5}}=0}{\displaystyle \mathrm {tf} ({\mathsf {''example''}},d_{1})={\frac {0}{5}}=0}</math>
-* <math>\mathrm {tf} ({\mathsf {''example''}},d_{2})={\frac {3}{7}}\approx 0.429</math>
+* <math>{\displaystyle \mathrm {tf} ({\mathsf {''example''}},d_{2})={\frac {3}{7}}\approx 0.429}{\displaystyle \mathrm {tf} ({\mathsf {''example''}},d_{2})={\frac {3}{7}}\approx 0.429}</math>
 |-
 |
-* <math>\mathrm {idf} ({\mathsf {''example''}},D)=\log \left({\frac {2}{1}}\right)=0.301</math>
+* <math>{\displaystyle \mathrm {idf} ({\mathsf {''example''}},D)=\log \left({\frac {2}{1}}\right)=0.301}{\displaystyle \mathrm {idf} ({\mathsf {''example''}},D)=\log \left({\frac {2}{1}}\right)=0.301}</math>
 |-
 |
-* <math>\mathrm {tfidf} ({\mathsf {''example''}},d_{1},D)=\mathrm {tf} ({\mathsf {''example''}},d_{1})\times \mathrm {idf} ({\mathsf {''example''}},D)=0\times 0.301=0</math>
+* <math>{\displaystyle \mathrm {tfidf} ({\mathsf {''example''}},d_{1},D)=\mathrm {tf} ({\mathsf {''example''}},d_{1})\times \mathrm {idf} ({\mathsf {''example''}},D)=0\times 0.301=0}{\displaystyle \mathrm {tfidf} ({\mathsf {''example''}},d_{1},D)=\mathrm {tf} ({\mathsf {''example''}},d_{1})\times \mathrm {idf} ({\mathsf {''example''}},D)=0\times 0.301=0}</math>
-* <math>\mathrm {tfidf} ({\mathsf {''example''}},d_{2},D)=\mathrm {tf} ({\mathsf {''example''}},d_{2})\times \mathrm {idf} ({\mathsf {''example''}},D)=0.429\times 0.301\approx 0.129</math>
+* <math>{\displaystyle \mathrm {tfidf} ({\mathsf {''example''}},d_{2},D)=\mathrm {tf} ({\mathsf {''example''}},d_{2})\times \mathrm {idf} ({\mathsf {''example''}},D)=0.429\times 0.301\approx 0.129}{\displaystyle \mathrm {tfidf} ({\mathsf {''example''}},d_{2},D)=\mathrm {tf} ({\mathsf {''example''}},d_{2})\times \mathrm {idf} ({\mathsf {''example''}},D)=0.429\times 0.301\approx 0.129}</math>
 |}