我正在使用python，我想浏览一个数据集并突出显示最常用的位置。在
这是我的数据集（但有300000多条记录）：Longitude Latitude
14.28586 48.3069
14.28577 48.30687
14.28555 48.30678
14.28541 48.30673
首先，我添加了一个密度列：
^{pr2}$
这是我用来增加每个记录的密度值的代码：for index in range(0,len(df)):
for index2 in range(index + 1, len(df)):
if df['Longitude'].loc[index] == df['Longitude'].loc[index2] and df['Latitude'].loc[index] == df['Latitude'].loc[index2]:
df['Density'].loc[index] += 1
df['Density'].loc[index2] += 1
print("match")
print(str(index) + "/" + str(len(df)))
上面的代码只是在dataframe中迭代，将第一条记录与数据集中的所有其他记录进行比较（内部循环），当找到匹配项时，它们的密度值都将递增。在
我想找到匹配的经纬度，并增加它们的密度值。在
代码显然非常慢，我相信Python会有一种很酷的技术来做这样的事情，有什么想法吗？在

 

 
I want to find the matching item from the below given list.My List may be super large.
 
The very first item in the tuple "N1_10" is duplicated and matched with another item in another array
 
tuple in 1st array in the ListA ('N1_10', 'N2_28')
 
tuple in 2nd array in the ListA ('N1_10', 'N3_98')
 
ListA = [[('N1_10', 'N2_28'), ('N1_35', 'N2_44')],
 
[('N1_22', 'N3_72'), ('N1_10', 'N3_98')],
 
[('N2_33', 'N3_28'), ('N2_55', 'N3_62'), ('N2_61', 'N3_37')]]
 
what I want for the output is
 
output --> [('N1_10','N2_28','N3_98') , .... and the rest whatever match one of the
 
key will get into same tuple]
 
If you guys think , changing the data structure of the ListA is better option , pls feel free to advise!
 
Thanks for helping out!
 
SIMPLIFIED VERSION
 
List A = [[(a,x),(b,k),(c,l),(d,m)],[(e,d),(a,p),(g,s)],[...],[...]....]
 
wantedOutput --> [(a,x,p),(b,k),(c,l),(d,m,e),(g,s).....]
 
解决方案
 
Update: After rereading your question, it appears that you're trying to create equivalence classes, rather than collecting values for keys. If
 
[[(1, 2), (3, 4), (2, 3)]]
 
should become
 
[(1, 2, 3, 4)]
 
, then you're going to need to interpret your input as a graph and apply a connected components algorithm. You could turn your data structure into an adjacency list representation and traverse it with a breadth-first or depth-first search, or iterate over your list and build disjoint sets. In either case, your code is going to suddenly involve a lot of graph-related complexity, and it'll be hard to provide any output ordering guarantees based on the order of the input. Here's an algorithm based on a breadth-first search:
 
import collections
 
# build an adjacency list representation of your input
 
graph = collections.defaultdict(set)
 
for l in ListA:
 
for first, second in l:
 
graph[first].add(second)
 
graph[second].add(first)
 
# breadth-first search the graph to produce the output
 
output = []
 
marked = set() # a set of all nodes whose connected component is known
 
for node in graph:
 
if node not in marked:
 
# this node is not in any previously seen connected component
 
# run a breadth-first search to determine its connected component
 
frontier = set([node])
 
connected_component = []
 
while frontier:
 
marked |= frontier
 
connected_component.extend(frontier)
 
# find all unmarked nodes directly connected to frontier nodes
 
# they will form the new frontier
 
new_frontier = set()
 
for node in frontier:
 
new_frontier |= graph[node] - marked
 
frontier = new_frontier
 
output.append(tuple(connected_component))
 
Don't just copy this without understanding it, though; understand what it's doing, or write your own implementation. You'll probably need to be able to maintain this. (I would've used pseudocode, but Python is practically as simple as pseudocode already.)
 
In case my original interpretation of your question was correct, and your input is a collection of key-value pairs that you want to aggregate, here's my original answer:
 
Original answer
 
import collections
 
clusterer = collections.defaultdict(list)
 
for l in ListA:
 
for k, v in l:
 
clusterer[k].append(v)
 
output = clusterer.values()
 
defaultdict(list) is a dict that automatically creates a list as the value for any key that wasn't already present. The loop goes over all the tuples, collecting all values that match up to the same key, then creates a list of (key, value_list) pairs from the defaultdict.
 
(The output of this code is not quite in the form you specified, but I believe this form is more useful. If you want to change the form, that should be a simple matter.)

 
SELECT
 某一列,
 COUNT( 某一列 )
 FROM
 表
 GROUP BY
 某一列
 HAVING
 COUNT( 某一列 ) 〉1
 这样查询出来的结果， 就是 有重复， 而且 重复的数量。
 

 
 
查询数据库中某一列有没有重复数据项：
 
 
select * from cd_stock where stock_bh in (select stock_bh from cd_stock group by stock_bh having count(stock_bh) >1 )
 
 
select * from cd_stock_item where id in (select id from cd_stock_item group by id having count(id) >1 )
 
 
转载于:https://www.cnblogs.com/bingege/p/8445142.html