Wednesday, January 31, 2018

ID generation



Related:
http://massivetechinterview.blogspot.com/2015/06/twitteridsnowflake-iteye.html
http://massivetechinterview.blogspot.com/2015/10/generate-unique-sequence-number-using.html
http://massivetechinterview.blogspot.com/2016/06/buttercola-fast-id-generator.html

https://www.quora.com/Scalability-Why-Is-database-ID-generation-a-bottleneck
Imagine the classic simple case of a single table with an auto-incrementing primary key (or, if you prefer, a SEQUENCE). Now imagine there are very many connections to the database server, each of which is inserting records on this table. At some point, those inserts must be serialised (because of the rules that the sequence of IDs must follow: sequential, no gaps). This is achieved by a lock which amounts to a critical section (i.e. only one connection can increment the sequence at a time), and every connection needs to acquire that lock before obtaining the next ID (and inserting a record).

 Many systems don't see Id generation as problem, which is because they don't have a large number of requests. Let's say you want to allocate an ID to each user and the ID should be unique and incremental by the registration date. Once you are receiving tons of registration requests every second or even millisecond, you will find it extremely easy to have duplicate IDs.

https://stackoverflow.com/questions/22883304/why-is-auto-increment-pattern-bad-when-scaling-in-mongodb
In order to guarantee that an auto-increment value is unique, the ID creation must occur on a single thread on a single host (even if multiple threads are used, the point of ID creation must block other threads). So, in a cluster of 100 servers, IDs must be created on 1 thread on 1 out the 100 servers. This not just a performance bottleneck, it is possible that the creation of 2 auto-increment IDs might block each other, which is the race condition noted in the quotation you've cited.
It should be noted that transactional RDBMS systems like Oracle and SQL Server have solved the race condition problem, but there is no solution to the performance bottleneck.
So: no, don't use auto-increment in non-primary keys if you anticipate the need to scale your system.

Clock synchronization

We ignored a crucial problem in the above analysis. In fact, there’s a hidden assumption that all ID generation servers have the same clock to generate the timestamp, which might not be true in distributed systems.
In reality, system clocks can drastically skew in distributed systems, which can cause our ID generators provide duplicate IDs or IDs with incorrect order. Clock synchronization is out of the scope of this discussion, however, it’s important for you to know such issue in this system. There are quite a few ways to solve this issue, check NTP if you want to know more about it.



Friday, January 26, 2018

Programmer Misc



https://www.quora.com/Why-do-you-need-a-big-screen-for-programming
And it doesn’t stop there. Especially if you are debugging graphical applications it really really nice if you can have the app you are debugging on a separate screen. It’s kind of annoying if you constantly have to switch windows back and forth and back and forth ……
Programming is about reading things three times as much as writing.

If you are a programmer, you usually have the following open at work
a) The IDE
b) A window in which you test your application
c) Web browser to look up online resources. Also, to keep track of CI/CD builds
d) Email client
e) Chat application
Out of these a) b) and c) need to looked at frequently. Frequentlt switching screens can he headache-inducing. Also, the bigger the IDE screen, the better. The more code you can see at onetime, the better it is to visualize it working in your head.


https://mp.weixin.qq.com/s/heFyKmlmhYz2dzCgPzRUiw
和大部分 app 一样,猿题库最早的时候使用的 MVC 架构。这个朴素的架构在早期发挥了他糙快猛的优良作用,但很快就把我们的 app 变得异常难维护。一个 VC 里面最长有 1k5 行代码,这对我来说是非常难以忍受的。
思考很久之后,秉承最小表现力原则,在尽可能不增加架构复杂度的基础上,心里默念没有什么问题是不能通过加一个中间件解决的思想,在原来的 MVC 架构上,增加了一些分层,使得各层的职责更明确,解耦更彻底:
M-S-DC-VC-VM-V
Model:数据模型
Service(Agent,ModelDataController): 数据逻辑
DataController: 界面逻辑,和 VC 一一对应
ViewController: 控件布局和一些胶水代码,和 DC 一一对应。
ViewModel:控件展示 Model
View: 控件
我认为,架构的目的首先在于统一,然后是效力。这个架构比起当时业界流传的各种新架构,最大的优势就是非常好理解,非常好实施,推广和切换的成本非常低。本质上来说这就是一个 MVC 的变种,没有双向绑定,没有不可控的第三方库,没有难以理解的概念。我在团队推广这个架构不到半年时间,就已经把项目里的大部分代码都切换到这个架构
https://juejin.im/post/5949ea4aa0bb9f006b0588df
•低效研发体系:把技术团队简单当资源使用,简单的任务派发。为什么要做这个需求,需求的价值是什么一概不知。
•高效研发体系:让技术团队觉得是在一起做一件有价值的事, 讨论清楚需求的前因后果。每次需求上线后,需求的效果有完整的数据跟踪,根据数据来总结本次项目的得失。
•低效研发体系:业务,产品和技术的观点不在一个维度上:业务的观点在业务拓展、收益上;产品的观点在需求、产品功能上;技术的观点又在产品特性,功能实现上。
•高效研发体系: 大家都聚焦在一个点上:”SHOW ME THE MONEY”,大家的沟通思路都围绕在提升产品收益或者降低营收成本,技术的所有工作都应该紧密围绕“提升收益”和“降低成本”这两个方向。
•低效研发体系: 需求评审会上,业务和产品唱“独角戏”,技术同学没有自己的想法。
•高效研发体系: 在需求评审会上,产品和技术会明确需求是否完整,确认产品功能的正常场景,是否形成闭环;异常场景的处理流程;产品细节是否考虑周全。



Tuesday, January 23, 2018

Useful websites for programmers



Papers
http://the-paper-trail.org/

the morning paper
https://blog.acolyer.org/

https://github.com/sdmg15/Best-websites-a-programmer-should-visit
https://www.infoq.com/presentations/

https://www.danylkoweb.com/Blog/top-10-web-sites-every-programmer-should-visit-on-a-daily-basis-HK
https://news.ycombinator.com/
dzone.com

Chinse
https://segmentfault.com/
https://www.zhihu.com/question/32002286

1. 开发者头条: toutiao.io/ 2. 伯乐在线: jobbole.com/ 3. 极客头条: geek.csdn.net/ 4. 掘金: juejin.im/ 5. SDK.CNsdk.cn/ 6. 深度开源:open-open.com/ 7. 推酷: tuicool.com/ 8. V2EX: v2ex.com/ 9. SegmentFault: segmentfault.com/ 10. InfoQ: infoq.com/ 11. CSDN: csdn.net/ 12. 51CTO: 51cto.com/ 13. 开源中国: oschina.net/

https://segmentfault.com


https://www.xttblog.com/?p=705

https://www.javacodegeeks.com/

http://www.importnew.com

GW
https://blog.jooq.org
https://www.infoq.com/java
http://www.infoq.com/cn
http://blog.cleancoder.com/uncle-bob

https://techfoco.com/tagcontents/Java

http://www.10tiao.com/
http://www.10tiao.com/author/index?authorId=249

https://techfoco.com/tagcontents/Java
https://www.tuicool.com/topics/11000072

https://www.javamex.com/
https://coderwall.com/t/java/popular

架构师之路
https://mp.weixin.qq.com/s/OlFKpcnBOgcPZmjvdzCCiA

Good blogs
http://afghl.github.io/

IG
-----
https://blog.takipi.com/
http://javarevisited.blogspot.sg/
https://www.javaworld.com/
https://javax0.wordpress.com/

http://geek.csdn.net/hotest

tech company blogs
https://engineering.instagram.com/

Blogs
https://bravenewgeek.com
https://github.com/iluwatar/java-design-patterns
https://github.com/kdn251/interviews
http://www.jdon.com/

http://www.raychase.net/ 四火的唠叨
http://blog.longjiazuo.com/

Blogs - Porgramming
https://henrikwarne.com


https://www.javacodegeeks.com/

Interview
http://blog.gainlo.co/
https://www.interviewbit.com/all-problem-list/

https://www.artima.com/intv/
http://c2.com/
http://wiki.c2.com/

RSS
https://feedly.com/

https://medium.com/
https://engineering.flosports.tv/best-company-engineering-blogs-on-medium-dd7451ee8168

News
https://slashdot.org/


Collection
https://github.com/sdmg15/Best-websites-a-programmer-should-visit
Hacker News and reddit/r/programming
https://becominghuman.ai/code-wonders-96d629bb8d8c


?
https://www.theserverside.com/
http://www.linkedkeeper.com/
http://www.nosqlnotes.com/
http://www.yegor256.com/
https://scalingsystems.com/

to-do 2 only
http://afeinberg.github.io/
http://www.puncsky.com/


http://www.deadcoderising.com

System desgin
https://www.linkedin.com/pulse/100-open-source-big-data-architecture-papers-anil-madan/
https://github.com/checkcheckzz/system-design-interview
http://www.puncsky.com/blog/2016/02/14/crack-the-system-design-interview

TODO:
https://docs.google.com/document/d/1dNKjHICogW5f94MKoBr8wDA3TASbhP0nrcy4eKiuA8Q/edit
https://www.quora.com/What-are-some-good-blogs-about-distributed-systems
https://www.allthingsdistributed.com
http://dbmsmusings.blogspot.com/

Chinese
http://www.ruanyifeng.com/blog/developer/
https://tech.meituan.com/archives
http://www.rowkey.me/
http://liangfei.me/archives/
https://github.com/aCoder2013/blog/issues

https://www.ysofters.com/
http://winterbe.com/blog/
https://blog.bcmeng.com/

Algorithms:
http://zxi.mytechroad.com/blog/

https://blog.jdriven.com/
http://yogoup.sinaapp.com/


Python
https://dbader.org/blog/

https://allegro.tech/
http://fahdshariff.blogspot.com/2011/04/writing-your-own-bash-completion.html

https://ryanstutorials.net

http://www.mokacoding.com/archive.html

https://technologyconversations.com/
http://sayhiai.com/index.php/page/2/

https://michaelheap.com/blog/
https://github.com/crossoverJie/Java-Interview
https://www.xilidou.com/
https://droidyue.com/blog/archives/
https://tonybai.com/articles/ - go

https://javarevisited.blogspot.com/
http://www.cnblogs.com/Zachary-Fan/

https://emmanuelbernard.com/blog/
https://makandracards.com/makandra

https://www.baeldung.com/
https://daniel.mitterdorfer.name/archive/
https://vmlens.com/blog-2/


Tech blogs
https://medium.com/@Pinterest_Engineering

架构师之路
https://www.w3cschool.cn/architectroad/

https://www.jdon.com/

SOLR
https://medium.com/@sarkaramrit2


https://byoungd.gitbook.io/english-level-up-tips/

https://smallbusinessprogramming.com/
https://peterdaugaardrasmussen.com/

https://piotrminkowski.wordpress.com
https://jiajunhuang.com/archive

https://mywiki.wooledge.org
https://guide.bash.academy/

SEI CERT Oracle Coding Standard for Java
https://wiki.sei.cmu.edu/confluence/display/java/SEI+CERT+Oracle+Coding+Standard+for+Java

https://www.alexecollins.com
https://dev.to/
http://allenlsy.com/archives.html

http://java-design-patterns.com/patterns/
https://samnewman.io/writing/

https://crossoverjie.top/
http://java-performance.info/
https://zhuanlan.zhihu.com/codehole
http://blog.xiayf.cn/archives.html
https://colobu.com/archives/
https://major.io

跨界架构师
https://toutiao.io/subjects/268968


http://callistaenterprise.se/blogg/

iTerm2



https://codeburst.io/8-must-know-terminal-commands-and-tips-for-productivity-mac-edition-95935dba3ebc
3) iTerm2 split screening

The hotkeys for horizontal split are: cmd + d

The hotkeys for vertical split are: cmd + shift+ d


https://dev.to/therealdanvega/new-macbook-setup-for-developers-2nma
brew cask install iterm2
  • Launch iTerm 2. Get the latest version at iterm2.com
  • Type CMD+i
  • Navigate to Colors tab
  • Click on Load Presets
  • Click on Import
  • Select the .itermcolors file(s) of the scheme(s) you'd like to use
  • Click on Load Presets and choose a color scheme
https://gist.github.com/squarism/ae3613daf5c01a98ba3a
Clear the screen/pane (when Ctrl + L won't work) + K (I use this all the time)
https://superuser.com/questions/1018391/iterm2-clear-scrollback-buffers-for-all-sessions
cmd+K will clear current session
https://elweb.co/making-iterm-2-work-with-normal-mac-osx-keyboard-shortcuts/
The ones that I needed most where ⌥← (Option-Left Arrow) to move left one word, ⌥→ (Option-Right Arrow) to move right one word


https://gitlab.com/gnachman/iterm2/issues/3717#note_1733467
[Restore last windows after restart]
Ensure Prefs>Advanced>Restore window contents at startup is set to Yes. This is the default, so I doubt it's changed. Also Make sure Close windows when quitting an app is off in System Prefs>General:

https://medium.com/@jessesrsmith/five-tips-for-iterm-91db83cf4d4e
This one can save a lot of time. To access everything you’ve pasted into iTerm, use ⌘⇧H and select the command you wish to paste again.

To create a new workspace, first cd into your project’s directory. To create a horizontal split, use ⌘⌥⇧H, or ⌘⌥⇧V for a vertical split. When creating a new pane, iTerm will ask what profile you want to use. After you have a few panes created, you can rearrange them by dragging with the mouse. Once your panes are in a preferred configuration, just hit ⌘⇧S to save the window arrangement. To launch your arrangement, use ⌘⇧R at anytime.
For those using laptops or smaller monitors, ⌘⇧⏎ will maximize/minimize the active pane. This is useful for examining log files in greater depth.

https://superuser.com/questions/333591/windows-open-maximized-by-default
Somehow most other software I use opens up maximized anyway, possibly because I use ShiftIt and Ctrl-Alt-Cmd M a lot (and most software seems to remember the previously used window size). The only thing I use a lot that didn't open maximimized was iTerm2.

You can get iTerm2 to open in a maximized window by going to Preferences -> Profiles -> Window and setting a high number of Columns and Rows

https://gitlab.com/gnachman/iterm2/issues/4162
Recent directories (including starred ones) are gone after closing and reopening iTerm2
Shell integration works fine and continuously adds directories to my "Recent directories" section in the toolbelt.
Starting another session in a new window perfectly shows the same recent and starred directories. Multiple sessions of the same profile keep in sync as expected. But closing all sessions and reopen a new one works fine only as long as I do not close iTerm itself (what I do regularly to install nightly builds).

I would love to keep the starred directories persistently.

urning on Prefs > General > Save copy/paste and command history

https://superuser.com/questions/93573/select-text-in-iterm-using-keyboard
You could also assign custom keys in Prefs -> Keys. (Source)

move selection forward by word
move selection backward by word
https://gist.github.com/squarism/ae3613daf5c01a98ba3a

https://www.iterm2.com/documentation-badges.html
\(session.username)@\(session.hostname)
Its initial value is defined in Preferences>Profiles>General>Badge and it can be changed by an iTerm2-proprietary escape sequence. It may also reference iTerm2- and user-defined variables.
https://www.iterm2.com/documentation-images.html

Open Quickly

If you have lots of sessions you can quickly find the one you're looking for with Open Quickly. Select the View > Open Quickly menu item (cmd-shift-O) and then enter a search query. You can search by tab title, command name, host name, user name, profile name, directory name, badge label, and more. Open Quickly also lets you create new tabs, change the current session's profile, and open arrangements. If you start your query with a / then that gives you a shortcut to various commands. Enter a query of / to see them.

Cursor Location Assistance

Never lose your cursor. An optional cursor guide highlights the entire row your cursor is on and Cursor Boost dims all colors other than the cursor to make it really stand out.

Undo Close

Undo closing sessions, tabs, and windows. If you close a session by accident, you get five seconds to hit Cmd-Z to undo it.

Tab Bar on Left

Have lots of tabs? Put the tab bar on the left. Quickly search them by recent commands, directories, current host name, profile name, and more with the Open Quickly feature.

https://www.iterm2.com/features.html

Captured Output

The next time you log in, shell integration will be enabled.

You will also have these commands:
imgcat filename
  Displays the image inline.
it2dl filename
  Downloads the specified file, saving it in your Downloads folder.

You can navigate marks with Cmd-Shift-Up and Down-arrow keys.

Alert on next mark

iTerm2 can show an alert box when a mark appears. This is useful when you start a long-running command. Select Edit>Marks and Annotations>Alert on next mark (Cmd-Opt-A) after starting a command, and you can go do something else in another window or tab. When the command prompt returns, a modal alert will appear, calling attention to the finished job.

Command status

The mark on a command line will turn red if a command fails. You can right click the mark to view its return code.
Command history popup
You can view and search the command history with Session>Open Command History... (Shift-Cmd-;).
Autocomplete
Commands in command history are also added to Autocomplete (Cmd-;). If Preferences>General>Save copy/paste history and command history to disk is enabled, then command history will be preserved across runs of iTerm2 (up to 200 commands per user/hostname).
Toolbelt
A command history tool may be added to the toolbelt by selecting Toolbelt>Command History.

Recent Directories popup
You can view and search your recently and frequently used directories in Session>Open Recent Directories... (Cmd-Opt-/).

download with scp  - need install shell integration on remote host  

Default characters considered part of word for selection
/-+\~_.
https://medium.com/@jessesrsmith/five-tips-for-iterm-91db83cf4d4e

4. Paste History

This one can save a lot of time. To access everything you’ve pasted into iTerm, use ⌘⇧H and select the command you wish to paste again.

2. Clickable Links

A simple one: hold the  key and click on any URL. iTerm will open the link in a new browser tab. This is great for opening localhost windows after a dev server starts. 
http://teohm.com/blog/working-effectively-with-iterm2/
  • clear buffer Cmd + k
  • toggle maximize window Cmd + Alt + =
  • toggle full screen Cmd + Enter
  • make font larger Cmd + +
  • make font smaller Cmd + -
Launch iTerm, open iTerm > Preferences or just Cmd + ,.

Open tab/pane with current working directory

Under Profiles tab, go to General subtab, set Working Directory to “Reuse previous session’s directory”.

Enable Meta key

To enable Meta key for Bash readline editing e.g. Alt + b to move to previous word, under Profilestab, go to Keys subtab, set Left option key acts as: to “+Esc”.

Hotkey to toggle iTerm2

Under Keys tab, in Hotkey section, enable “Show/hide iTerm2 with a system-wide hotkey” and input your hotkey combination, e.g. I use Ctrl + Shift + L.
https://medium.com/@hlung/using-smart-selection-in-iterm2-to-make-asana-task-id-or-any-text-clickable-dddbbbe2bbe8
One thing I notice in iTerm2 is if we hold  and hover around texts, URLs will be highlighted and clickable. It would be nice if I can do this to other text as well. It turns out there’s a feature in iTerm2 that can help us — it’s called Smart Selection.

https://www.iterm2.com/documentation-smart-selection.html
You should be able to make iTerm.app your default terminal by selecting "Make Default" from the menu bar.


iTerm2 是最常用的终端应用,是 Terminal 应用的替代品。提供了诸如Split Panes一群实用特性。它默认的黑色背景让我毫不犹豫的抛弃了 Terminal。
安装:
brew cask install iterm2
感谢 brew-cask,我们可以通过命令行自动安装 iTerm2 了。
在终端里,除了可以用⌃E等快捷键(详见其他快捷键)之外,还可以使用⌥B⌥F等快捷键(具体可以参考这里)。前提是这样设置一下:
选择Iterm菜单 > Preferences > Profiles,选择你在使用的 Profile(默认是Default),在Keys标签页中把Left option (⌥) key acts asRight option (⌥) key acts as都设置成+ESC
在打开新的窗口/标签页的时候,默认情况下新窗口总是 HOME 目录,还需要我每次敲命令才能进入工作目录。如果想要这个新窗口在打开的时候就自动进入工作目录,需要如下设置:
选择Iterm菜单 > Preferences > Profiles,选择你在使用的 Profile(默认是Default),在General标签页中的Working Directory部分中选择Reuse previous seesion's directory
至此,Terminal 应用已经出色的完成了其历史使命。后面命令行就交给 iTerm2 啦。
在 iTerm2 中双击会自动选中对应的词,三击会选中对应的整行。选中的内容会自动进入剪贴板,不需要再按⌘C复制。
http://apple.stackexchange.com/questions/63000/how-to-highlight-errors-and-warnings-on-iterm-terminal-output
iTerm supports coloring of console output based on a set of regular expressions. You can set them up in Preferences > Profiles > Advanced > Triggers > Edit.
Here is my current set of regexes:
(?i:.*error.*)                     // Yellow on Black
(?i:.*(warning|warn).*)            // Orange on Black
(?i:.*FATAL.*)                     // White on Red
iTerm regexes in a profile’s Triggers

Labels

Review (572) System Design (334) System Design - Review (198) Java (189) Coding (75) Interview-System Design (65) Interview (63) Book Notes (59) Coding - Review (59) to-do (45) Linux (43) Knowledge (39) Interview-Java (35) Knowledge - Review (32) Database (31) Design Patterns (31) Big Data (29) Product Architecture (28) MultiThread (27) Soft Skills (27) Concurrency (26) Cracking Code Interview (26) Miscs (25) Distributed (24) OOD Design (24) Google (23) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Interview Q&A (20) System Design - Practice (20) Tips (19) Algorithm (17) Company - Facebook (17) Security (17) How to Ace Interview (16) Brain Teaser (14) Linux - Shell (14) Redis (14) Testing (14) Tools (14) Code Quality (13) Search (13) Spark (13) Spring (13) Company - LinkedIn (12) How to (12) Interview-Database (12) Interview-Operating System (12) Solr (12) Architecture Principles (11) Resource (10) Amazon (9) Cache (9) Git (9) Interview - MultiThread (9) Scalability (9) Trouble Shooting (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Cassandra (8) Company - Uber (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Design (7) Interview Corner (7) JVM (7) Java Basics (7) Kafka (7) Mac (7) Machine Learning (7) NoSQL (7) C++ (6) Chrome (6) File System (6) Highscalability (6) How to Better (6) Network (6) Restful (6) CareerCup (5) Code Review (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) Python (5)

Popular Posts