Posts Tagged ‘nosql’

Performance Benchmark — Storm v Spark v Flink (Re-post)

https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at

Hive: Make CLI output files comma delimited

bash >> hive -e ‘select * from some_Table’ | sed ‘s/[\t]/,/g’ > outputfile.txt

Here [\t] means Control+V and then the tab button, i.e.
sed ‘s//,/g’

Example:

[user@server]$ hive -e "use dbname ; select * from tablename" | sed ‘s/ /,/g’ > kpi_event_jan8.csv

HiveQL: SQL Connector

 
To bridge the difference between SQL and HiveQL, we have the SQL Connector
feature to translate standard SQL-92 queries into equivalent HiveQL queries. The SQL
Connector performs syntactical translations and structural transformations. For
example:
1. Quoted Identifiers
HiveQL uses back-quote while SQL uses double quote when quoting identifiers.
Even when a driver reports the back-quote as the quote character, some
applications still generate double quoted identifiers.
2. Table Aliases
HiveQL does not support the AS keyword between a table reference and its
alias.
3. JOIN, INNER JOIN and CROSS JOIN
SQL INNER JOIN and CROSS JOIN syntax is translated to HiveQL JOIN syntax.
4. TOP N/LIMIT
SQL TOP N queries are transformed to HiveQL LIMIT queries.

Hbase: Disable and Drop

Disable and drop your table to clean up:

hbase(main):012:0> disable 'test'
0 row(s) in 1.0930 seconds
hbase(main):013:0> drop 'test'
0 row(s) in 0.0770 seconds 

Exit the shell by typing exit.

hbase(main):014:0> exit

Quick Tip: HBase Scan (a simple example)

hbase(main):008:0> scan ‘test’
ROW                   COLUMN+CELL
 row1                 column=cf:a, timestamp=1381963930588, value=value1
 row2                 column=cf:b, timestamp=1381963944569, value=value2
 row3                 column=cf:c, timestamp=1381963957538, value=value3

3 row(s) in 0.1270 seconds

Quick Tip: How to Connect to a Running Session (shell) in HBase in 1 Line

./bin/hbase shell

Quick-Tip: How to Search SQL/NoSQL/Postgres Information Schema in 1 Line

SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_NAME LIKE ‘%booty%’

This also works for table_name!

Quick-Tip: Connect to a ParAccell (aka PaDB) DB from Bash in 1 Line

psql -h xen-01.yourhostname.com –username paraccel hsa -p 5439

Quick-Tip: Cross-DB Hive Joins in 1 Line

select * from db.table f, db2.table2 j WHERE f.id = j.id;

Quick Tip: Query Hive on a Bash CLI and Write Results to a File in 1 Line

 hive -S -e “SELECT * FROM your_table” > ~/outfile.tsv