I'm gathering recent cross posts of about a 1000 popular subreddits, But the code takes a while to run.. How do I speed up this process? Please help!
submission_data = []
sub_count = 0
for sub in popsublist:
count = 0
sub_count += 1
print('============================')
print('subs-looped count:',sub_count)
print('current sub:',sub)
print('============================')
sub_loop = 0
for post in reddit.subreddit(sub).hot(limit=500):
sub_loop += 1
print("posts-looped count",sub_loop)
if hasattr(post, "crosspost_parent"):
count += 1
print('posts-loop count [ADDED!]:',count)
op = reddit.submission(id=post.crosspost_parent.split("_")[1]).subreddit
submission_data.append({
'SOURCE_SUB': str(post.subreddit),
'TARGET_SUB': str(op),
'POST_ID': str(post.id),
'POST_TITLE': str(post.title),
'POST_DATE': datetime.utcfromtimestamp(int(post.created_utc)).strftime('%Y-%m-%d %H:%M:%S'),
'POST_LINK': str('http://www.reddit.com'+post.permalink),
'POST_SCORE': post.score,
'POST_NSFW': post.over_18,
})
This is my code. How do I accelerate this process?