タグ
- すべて
- *あとで (8)
- *あとで読む (40)
- .net (2)
- Deferred (1)
- MPLUG-DOCOWL (1)
- PACS (1)
- PlantUML (1)
- PlazmaDB (1)
- PubSub (1)
- SPDY (2)
- UIスレッド (1)
- WebIntents (1)
- accumulo (1)
- actian (1)
- activerecord (2)
- actor (9)
- actormodel (2)
- aerospike (3)
- agile (17)
- ai (6)
- airflow (1)
- ajax (2)
- akka (22)
- algorithm (18)
- alignment (1)
- ami (2)
- analysis (1)
- android (26)
- angular (1)
- angularjs (8)
- ansible (25)
- ant (1)
- apache (6)
- api (2)
- apt (1)
- arangodb (3)
- arangondb (1)
- architecture (1)
- arquillian (1)
- arrow (24)
- asakusa (3)
- asciidoc (1)
- asm (1)
- async (1)
- athena (1)
- atlas (1)
- aufs (1)
- aurora (9)
- auroradb (1)
- avro (2)
- aws (131)
- awscli (1)
- azure (5)
- bash (4)
- batch (1)
- bdb (5)
- bdd (1)
- benchmark (2)
- bff (1)
- bi (4)
- bigdata (1)
- bigquery (4)
- bigtable (3)
- bitcoin (1)
- bloom filter (5)
- bloomfilter (1)
- boltdb (1)
- book (13)
- books (1)
- bootstrap (9)
- bot (1)
- boto (1)
- browser (1)
- bsd (1)
- bson (3)
- btree (8)
- buck (1)
- budger (1)
- bundler (2)
- cache (6)
- calcite (10)
- capistrano (1)
- capybara (1)
- cascading (3)
- case class (1)
- cassandra (22)
- catalyst (1)
- cauchdb (2)
- cd (2)
- cdc (1)
- centos7 (1)
- cgroups (2)
- chat (1)
- chef (5)
- chef-solo (1)
- chrome (8)
- ci (8)
- citus (1)
- clickhouse (3)
- clojure (3)
- cloud (3)
- cloud-init (1)
- cloudera (2)
- cloudsearch (1)
- cloudwatch (2)
- cluster (1)
- cockroarchdb (1)
- codegen (1)
- codereview (1)
- coherence (2)
- color (1)
- columnar (8)
- commons (1)
- commons-logging (2)
- compiler (1)
- compression (1)
- concurrency (8)
- concurrent (9)
- config (1)
- consistency (2)
- consul (29)
- container (7)
- continuation (1)
- cookie (1)
- coreos (5)
- cors (1)
- couchbase (4)
- couchdb (25)
- cowboy (2)
- cpu (3)
- cqrs (1)
- cron (3)
- crossvalidation (3)
- cucumber (3)
- cwt2017 (1)
- cygwin (1)
- cypress (2)
- daas (1)
- dalvikvm (1)
- dart (1)
- dask (4)
- data (3)
- database (28)
- databricks (6)
- databus (1)
- datamanagement (4)
- datamining (1)
- dataops (1)
- datomic (5)
- db (5)
- dbcp (2)
- dbm (1)
- dbms (11)
- dci (3)
- ddd (7)
- debezium (4)
- debian (1)
- debug (5)
- deeplearning (1)
- degitalocean (1)
- delta (3)
- dempsy (1)
- deploy (3)
- derby (2)
- design (3)
- dev (3)
- development (10)
- devops (8)
- diff (1)
- digitalocean (1)
- disque (1)
- distel (1)
- distribute (1)
- django (1)
- dns (2)
- docker (121)
- dockr (1)
- docomo (1)
- dokku (1)
- dotcloud (1)
- drbd (1)
- drdb (1)
- dremel (4)
- drill (10)
- drone (1)
- dropwatch (1)
- dropwizard (1)
- druid (1)
- dsl (1)
- dstat (1)
- dwh (4)
- dynamo (7)
- dynamodb (14)
- e2 (1)
- e2e (2)
- ebook (3)
- ebs (2)
- ec2 (5)
- eclipse (18)
- ecs (3)
- egit (2)
- elasticache (4)
- elasticsearch (13)
- elb (6)
- electron (1)
- elixir (1)
- emacs (29)
- embulk (3)
- emma (1)
- emr (29)
- epub (1)
- erlang (58)
- espresso (1)
- etcd (3)
- etl (4)
- eunit (1)
- excel (1)
- explain (1)
- extension (2)
- fabric (7)
- facebook (1)
- filodb (1)
- findbugs (6)
- fitness (2)
- flash (2)
- flask (4)
- flatbuffers (1)
- fluentd (27)
- flume (1)
- food (1)
- forestdb (1)
- functional (2)
- g1gc (5)
- gandiva (2)
- ganglia (3)
- gc (13)
- gce (2)
- gcp (1)
- gdb (1)
- gem (1)
- gems (3)
- generics (1)
- ghe (1)
- gist (1)
- git (37)
- git-flow (1)
- giter8 (1)
- github (35)
- glue (2)
- gmail (3)
- gnu (2)
- go (4)
- golang (11)
- google (13)
- googleapps (1)
- googleglass (3)
- gossip (1)
- gpu (7)
- graal (9)
- graalvm (9)
- gradle (2)
- grafana (2)
- grails (1)
- graphite (4)
- groovy (3)
- growthforecast (1)
- grub (1)
- gtd (1)
- guard (1)
- guava (1)
- hadoop (74)
- hana (2)
- hanoidb (1)
- haproxy (6)
- hardware (1)
- hash (4)
- hashicorp (2)
- haskell (3)
- hazelcast (2)
- hbase (18)
- hcj2016 (1)
- hdfs (11)
- healthcare (32)
- heroku (7)
- hipchat (3)
- hive (40)
- hotspot (3)
- http (5)
- hudi (2)
- ignite (3)
- immutableinfrastructure (2)
- impala (14)
- infrastructure (3)
- infrataster (2)
- innodb (5)
- intellij (2)
- ios (2)
- ip (2)
- jackson (6)
- java (186)
- javascript (24)
- jdbc (6)
- jdk (3)
- jekyll (2)
- jenkins (31)
- jmeter (2)
- jmx (3)
- job (5)
- jobqueue (2)
- jquery (3)
- json (20)
- junit (3)
- jupyternotebook (8)
- jvm (65)
- k8s (4)
- kafka (34)
- kaggle (7)
- kernel (7)
- kindle (3)
- kotlin (3)
- kubernetes (12)
- kudu (4)
- kvm (3)
- kvs (20)
- kyototycoon (6)
- lambda (5)
- lamportclock (2)
- leveldb (8)
- life (2)
- linux (58)
- lisp (2)
- llm (4)
- log (6)
- log4j (7)
- logging (4)
- looker (2)
- lsm (2)
- lsm-tree (3)
- lua (2)
- lucene (4)
- lxc (3)
- machinelearning (4)
- macro (2)
- management (7)
- mapreduce (2)
- mariadb (2)
- markdown (3)
- maven (10)
- maven2 (2)
- memcached (6)
- memory (4)
- memsql (3)
- mercurial (6)
- mesos (2)
- microservice (5)
- microservices (2)
- migration (2)
- mnesia (2)
- mockito (6)
- mongodb (14)
- monitoring (5)
- mruby (3)
- mysql (106)
- nagios (5)
- neo4j (3)
- netty (10)
- network (4)
- nginx (4)
- nio (11)
- node.js (6)
- nodejs (3)
- nosql (20)
- octopress (8)
- olap (3)
- openjdk (3)
- optional (2)
- optiq (7)
- oracle (20)
- orcfile (2)
- oss (8)
- otto (2)
- paas (2)
- pacemaker (5)
- packer (7)
- pandas (13)
- parallel (3)
- parquet (23)
- paxos (3)
- performance (13)
- perl (4)
- phoenix (2)
- play (4)
- pmd (4)
- podcast (59)
- poi (2)
- postgresql (38)
- presto (9)
- profile (4)
- programming (19)
- proxy (3)
- python (45)
- quantum (2)
- quasar (3)
- raft (2)
- rails (18)
- rcfile (2)
- rdbms (11)
- rds (18)
- reactive (4)
- rebar (2)
- redash (3)
- redis (43)
- redshift (16)
- refactoring (2)
- regex (2)
- rest (5)
- review (2)
- riak (11)
- rocksdb (4)
- route53 (2)
- rspec (5)
- ruby (26)
- rust (3)
- s2dao (2)
- s3 (21)
- sbt (2)
- scala (65)
- scrum (7)
- security (13)
- seda (4)
- selenium (5)
- semaphore (2)
- serf (13)
- serialize (2)
- serverspec (11)
- shell (6)
- slf4j (2)
- softlayer (2)
- solr (12)
- spark (90)
- sphinx (2)
- spring (8)
- sql (11)
- sqlserver (3)
- sqoop (5)
- ssd (4)
- ssh (10)
- ssl (5)
- sstable (2)
- stash (2)
- stlite (2)
- storage (2)
- storm (7)
- strace (2)
- stream (7)
- sublimetext (4)
- subversion (2)
- systemd (2)
- tableau (2)
- tcp (3)
- tdd (4)
- techfirm (20)
- template (3)
- terracotta (5)
- terraform (9)
- test (43)
- tez (2)
- thrift (2)
- timescaledb (2)
- tls (2)
- tmux (3)
- tomcat (4)
- transaction (4)
- tree (2)
- trie (3)
- truffle (4)
- twisted (2)
- twitter (3)
- typescript (2)
- ubuntu (4)
- unikernels (2)
- unix (6)
- vaex (2)
- vagrant (14)
- vault (2)
- vectorclock (2)
- vertica (4)
- vim (7)
- virtualbox (2)
- vision (1)
- vlm (1)
- voldemort (3)
- voltdb (4)
- vpc (2)
- web (10)
- websocket (12)
- windows (9)
- work (33)
- wsl (2)
- xen (3)
- xgboost (2)
- yarn (3)
- zabbix (6)
- zookeeper (4)
- あとで読む (20)
- 川口 (2)
- 後で読む (2)
- java (186)
- aws (131)
- docker (121)
- mysql (106)
- spark (90)
- hadoop (74)
- jvm (65)
- scala (65)
- podcast (59)
- erlang (58)
cascadingに関するmsyktのブックマーク (3)
-
msykt 2015/04/26
S3にログを書き込むSecorというツールが気になったのと、SparkとMemSQLの連携が面白そう- log
- kafka
- singer
- s3
- secor
- spark
- cascading
- memsql
リンク -
Cascading使ってみた
久々の更新になってしまった。前回Google App Engine Pipeline APIを紹介したが、 その中でCascadingというキーワードが出てきたと思う。 今回はずっと気になっていたCascadingとやらHadoopの関連プロジェクトについて触れてみたい。 ただWordCountをやってみるのはもう飽きたのでw 今回はより実用的にApacheログの解析をやってみたい。 下記のログファイルを解析に使う。 Cascadingアプリケーションのソースは下記 処理の内容について、細かく説明するのはまた次回以降。（まだ調査、勉強中です。。） 大まかに処理の流れを追うと入力データ（ソース）から必要な部分を抽出し、 "ip", "time", "method", "event", "status", "size"という項目をもつタプル（データベースの行やレコードとよく似ている）に整形する
-
Cascadingサンプル：WordCount(Hishidama's Hadoop Cascading WordCount Memo)
WordCountサンプル CascadingでHadoopチュートリアルのWordCountと同様の処理を行うプログラム。 package jp.hishidama.hadoop.cascading.wordcount; import java.util.StringTokenizer; import org.apache.hadoop.fs.Path; import cascading.flow.Flow; import cascading.flow.FlowConnector; import cascading.flow.FlowProcess; import cascading.operation.Aggregator; import cascading.operation.AggregatorCall; import cascading.operation.BaseOperati
-
1
公式Twitter
- @HatenaBookmark
リリース、障害情報などのサービスのお知らせ
- @hatebu
最新の人気エントリーの配信
処理を実行中です
キーボードショートカット一覧
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
設定を変更しましたx