Friday, November 27, 2015

leetcode面试题19 最常访问IP



leetcode面试题19 最常访问IP
给你一个海量的日志数据,提取出某日访问网站次数最多的IP地址。

答:

将日志文件划分成适度大小的M份存放到处理节点。
每个map节点所完成的工作:统计访问百度的ip的出现频度(比较像统计词频,使用字典树),并完成相同ip的合并(combine)。
map节点将其计算的中间结果partition到R个区域,并告知master存储位置,所有map节点工作完成之后,reduce节点首先读入数据,然后以中间结果的key排序,对于相同key的中间结果调用用户的reduce函数,输出。
扫描reduce节点输出的R个文件一遍,可获得访问网站度次数最多的ip。
面试官角度:
该问题是经典的Map-Reduce问题。一般除了问最常访问,还可能会问最常访问的K个IP。一般来说,遇到这个问题要先回答Map-Reduce的解法。因为这是最常见的解法也是一般面试官的考点。如果面试官水平高一点,要进一步问你有没有其他解法的话,该问题有很多概率算法。能够在极少的空间复杂度内,扫描一遍log即可得到Top k Frequent Items(在一定的概率内)。有兴趣的读者,可以搜搜"Sticky Sampling","Lossy Counting"这两个算法。
Read full article from leetcode面试题19 最常访问IP

Labels

Review (572) System Design (334) System Design - Review (198) Java (189) Coding (75) Interview-System Design (65) Interview (63) Book Notes (59) Coding - Review (59) to-do (45) Linux (43) Knowledge (39) Interview-Java (35) Knowledge - Review (32) Database (31) Design Patterns (31) Big Data (29) Product Architecture (28) MultiThread (27) Soft Skills (27) Concurrency (26) Cracking Code Interview (26) Miscs (25) Distributed (24) OOD Design (24) Google (23) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Interview Q&A (20) System Design - Practice (20) Tips (19) Algorithm (17) Company - Facebook (17) Security (17) How to Ace Interview (16) Brain Teaser (14) Linux - Shell (14) Redis (14) Testing (14) Tools (14) Code Quality (13) Search (13) Spark (13) Spring (13) Company - LinkedIn (12) How to (12) Interview-Database (12) Interview-Operating System (12) Solr (12) Architecture Principles (11) Resource (10) Amazon (9) Cache (9) Git (9) Interview - MultiThread (9) Scalability (9) Trouble Shooting (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Cassandra (8) Company - Uber (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Design (7) Interview Corner (7) JVM (7) Java Basics (7) Kafka (7) Mac (7) Machine Learning (7) NoSQL (7) C++ (6) Chrome (6) File System (6) Highscalability (6) How to Better (6) Network (6) Restful (6) CareerCup (5) Code Review (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) Python (5)

Popular Posts