Kaggle API사용하기

https://paiai.tistory.com/30 참고

Jupyter Notebook 에서 Kaggle API 설치하고 사용하기

공식 문서 링크 : Official Kaggle API

Kaggle 링크 : https://www.kaggle.com/

1. 준비사항

kaggle 사이트에 접속해서 Profile > account tab > Create API Token => kaggle.json 파일 다운로드

Account 탭을 클릭한 후 하단에 Create API Token 버튼을 클릭하면 kaggle.json 파일이 다운로드된다. 

2. Jupyter Notebook 실행

Kaggle API 설치

!pip install kaggle

Collecting kaggle
  Downloading kaggle-1.0.5.tar.gz
Requirement already satisfied: urllib3>=1.15 in /Users/anaconda3/lib/python3.6/site-packages (from kaggle)
Requirement already satisfied: six>=1.10 in /Users/anaconda3/lib/python3.6/site-packages (from kaggle)
Requirement already satisfied: certifi in /Users/anaconda3/lib/python3.6/site-packages (from kaggle)Requirement already satisfied: python-dateutil in /Users/anaconda3/lib/python3.6/site-packages (from kaggle)
Building wheels for collected packages: kaggle
  Running setup.py bdist_wheel for kaggle ... done
  Stored in directory: /Users/Library/Caches/pip/wheels/56/4a/b5/93df30500029d78108fdfaace4f08fb881d27903f795a6367e
Successfully built kaggle
Installing collected packages: kaggle
Successfully installed kaggle-1.0.5

!pip show kaggle

Name: kaggle
Version: 1.0.5
Summary: Kaggle API
Home-page: https://github.com/Kaggle/kaggle-api
Author: Kaggle
Author-email: support@kaggle.com
License: Apache 2.0
Location: /Users/anaconda3/lib/python3.6/site-packages
Requires: certifi, urllib3, python-dateutil, six

Kaggle API가 설치된 위치를 확인한 후 다운로드 한 kaggle.json 파일을 이동시킨다

!kaggle config path

Your files will be downloaded to /Users/.kaggle

커맨드에서 실행 - kaggle.json 파일을 [kaggle config path] 아래에 위치하도록 이동한 후에 권한 설정

$ mv Downloads/kaggle.json /Users/.kaggle/.
$ chmod 600 /Users/.kaggle/kaggle.json

(on Windows in the location C:\Users\.kaggle\kaggle.json)
(Mac이나 Linux는 사용자(user)에 있음 ~ 기본폴더)

Commands

kaggle competitions {list, files, download, submit, submissions}
kaggle datasets {list, files, download}
kaggle config {path}

# 도움말 보기

!kaggle competitions list -h

usage: kaggle competitions list [-h] [-p PAGE] [-s SEARCH] [-v]

optional arguments:
  -h, --help            show this help message and exit
  -p PAGE, --page PAGE  page number
  -s SEARCH, --search SEARCH
                        term(s) to search for
  -v, --csv             print in CSV format
                        (if not set print in table format)

# health 와 관련된 경진대회 리스트 보기

!kaggle competitions list -s health

ref                                        deadline             category      reward  teamCount  userHasEntered  
-----------------------------------------  -------------------  --------  ----------  ---------  --------------  
hhp                                        2013-04-04 07:00:00  Featured    $500,000       1353           False  
diabetic-retinopathy-detection             2015-07-27 23:59:00  Featured    $100,000        661           False  
ultrasound-nerve-segmentation              2016-08-18 23:59:00  Featured    $100,000        923           False  
melbourne-university-seizure-prediction    2016-12-01 23:59:00  Research     $20,000        478           False  
msk-redefining-cancer-treatment            2017-10-02 23:59:00  Research     $15,000       1386           False  
second-annual-data-science-bowl            2016-03-14 23:59:00  Featured    $200,000        192           False  
intel-mobileodt-cervical-cancer-screening  2017-06-21 23:59:00  Featured    $100,000        848           False  
data-science-bowl-2017                     2017-04-12 23:59:00  Featured  $1,000,000       1972           False  
datasciencebowl                            2015-03-16 23:59:00  Featured    $175,000       1049           False  

# 튜토리얼과 관련된 경진대회 리스트 보기

!kaggle competitions list -s tutorial

ref                                     deadline             category            reward  teamCount  userHasEntered  
--------------------------------------  -------------------  ---------------  ---------  ---------  --------------  
word2vec-nlp-tutorial                   2015-06-30 23:59:00  Getting Started  Knowledge        578            True  
titanic                                 2020-04-07 00:00:00  Getting Started  Knowledge       9543            True  
street-view-getting-started-with-julia  2017-01-07 00:00:00  Getting Started  Knowledge         56           False  
spooky-author-identification            2017-12-15 23:59:00  Playground         $25,000       1244           False  

# 타이타닉 경진대회 데이터셋 다운로드

!kaggle competitions download -c titanic

train.csv: Downloaded 60KB of 60KB
test.csv: Downloaded 28KB of 28KB
gender_submission.csv: Downloaded 3KB of 3KB

# 기본으로 설정된 다운로드 경로

%ls ~/.kaggle/competitions/titanic/

gender_submission.csv  test.csv               train.csv

# 다운로드 경로를 설정해서 다운로드 받는 방법

!kaggle competitions download -c titanic -p ./kaggle_api_data/titanic

train.csv: Downloaded 60KB of 60KB
test.csv: Downloaded 28KB of 28KB
gender_submission.csv: Downloaded 3KB of 3KB

# 다운로드 된 파일 확인

!kaggle competitions files -c titanic

name                   size  creationDate         
---------------------  ----  -------------------  
train.csv              60KB  2013-06-28 13:40:25  
test.csv               28KB  2013-06-28 13:40:24  
gender_submission.csv   3KB  2017-02-01 01:49:18  

출처: https://paiai.tistory.com/30 [YOLO :)]