Skip to main content

Tech Enthusiast Weekly(2024-07-12) : Introducing Crawlee for Python: A Reliable Web Scraping and Browser Automation Tool

Introducing Crawlee for Python: A Reliable Web Scraping and Browser Automation Tool​

Introducing Crawlee for Python: A Reliable Web Scraping and Browser Automation Tool Introducing Crawlee for Python: A Reliable Web Scraping and Browser Automation Tool

Crawlee for Python: a new open-source tool. It excels at web scraping and browser automation. Quickly gathers data, stores it securely, dodges blocks. Uses headless browsers and smart proxy rotation.

Web scraping: extracting data from websites. Headless browsers: web browsers without a GUI, used for automation. Proxy rotation: changing IP addresses to avoid detection during data extraction.

Scores

Tech News​

DotaMath: Advanced Mathematical Reasoning with Code Assistance​

DotaMath, a new approach to complex math problems, breaks them down into simpler parts. It uses code to solve these parts, checks its work, and corrects mistakes. Trained on a large dataset of math questions and answers, DotaMath models perform well on tests, even outperforming other models. The method shows promise for tackling tough math challenges. Code is available online for anyone to use.

BM25S: Accelerating Lexical Search with Eager Sparse Scoring​

BM25S is a new Python tool. It accelerates search by 500 times, outperforming popular Python and Java tools. It achieves this by storing search scores in a specialized matrix during setup. This method makes it exceptionally fast. It also adeptly handles various types of BM25 searches. The code is available online for anyone to use.

"China showcases AI prowess at Shanghai forum"​

'China showcases AI prowess at Shanghai forum'

Chinese Premier Li Keqiang champions AI development at Shanghai forum, stressing global collaboration for safety and reliability. China's AI industry surpasses $68.7 billion, with top tech firms showcasing cutting-edge products. President Xi Jinping's goal is for China to lead AI by 2030. U.S. restrictions on AI chip exports to China have fueled domestic investment. Chinese government supports AI development with new funds and supervision of tech giants. Competition among AI companies intensifies in the domestic market. OpenAI's ChatGPT app success prompts global investment frenzy. China pushes for self-sufficiency in advanced technology, emphasizing safe and controllable AI for mankind's benefit.

Quark Search Engine upgrades to version 7.0, introducing AI services.​

Quark Search Engine has upgraded to version 7.0, introducing the "Super Search Box." New features include AI writing and document summarization. Usage during the college entrance examination season exceeded 100 million times. Quark's PC version has launched on Windows, integrating browsers, cloud storage, and more. Technical director Jiang Guanjun stated that the hallucination rate of the answer content has been reduced to 5%.

Hallucination Rate: Refers to the ratio of errors or inaccurate information in AI-generated content. Quark has significantly reduced this rate through large-scale model optimization.

EU Closes Apple Pay Antitrust Probe with Binding Commitments​

The EU closed its Apple Pay antitrust probe. Apple agreed to open NFC tech for rival wallets by July 25. This allows competitors to offer "tap and go" payments on iPhones and use Face ID, Touch ID, and passcodes for security.

Apple must allow users to set third-party wallets as default. The EU initially probed Apple Pay in 2020, focusing on NFC access. In 2022, the EU found that Apple blocked rivals from using NFC, stifling competition.

Apple proposed changes for 2024, including free access to NFC via APIs, excluding the secure element chip but offering "Host Card Emulation (HCE)" mode for secure transactions.

The EU refined Apple's commitments after industry feedback, with key changes including removing the need for a Payment Service Provider (PSP) license and adapting HCE to industry standards.

Apple's commitments exceed new EU rules, the Digital Markets Act (DMA). These pledges bind Apple for ten years, with potential penalties for non-compliance.

Apple's statement confirms they will enable NFC payments for various services while maintaining Apple Pay's security.

Helsing Raises $487M for AI Defense Expansion in Baltic Region​

Helsing, a defense AI firm, has just raised $487 million. They are expanding into Estonia with the aim of strengthening defenses against Russia. This move aligns with NATO's concerns over the Russian invasion of Ukraine.

Helsing's technology enhances drone and jet fighter capabilities and assists in battlefield decision-making. They have partnered with Airbus and several defense ministries.

Gundbert Scherf, co-CEO of the company, views their technology as essential for defending European democracies. Estonia's Prime Minister, Kaja Kallas, welcomes Helsing's presence.

Torsten Reil, Helsing's co-CEO, emphasizes the urgency to create a "capability gap" to deter and defend against potential broader threats.

The firm utilizes its own compute resources and some from unnamed third parties for security reasons. They are also working on Project Centaur, an AI for air combat that employs reinforcement learning.

This latest funding values Helsing at approximately $5.4 billion. Investors include Spotify's Daniel Ek and Saab AB.

Defense technology is gaining momentum with Western investors, reflecting the increase in defense budgets. Helsing stands out as a significant European player in this field.

OSIRIS-REx Asteroid Sample: Clues to Life's Origins​

OSIRIS-REx Asteroid Sample: Clues to Life's Origins

NASA's OSIRIS-REx mission successfully returned 121.6 grams of asteroid Bennu. Initial analysis indicates that the sample is abundant in carbon, nitrogen, and organic compounds—essential elements for life as we understand it. The sample also contains pristine magnesium-sodium phosphate, a first for any meteorite.

Bennu's composition suggests a history involving water, possibly originating from a small, ancient oceanic world. This discovery prompts questions about how these elements aggregated and provides insights into Bennu's past.

The asteroid's elements closely match those of the Sun, preserved since the birth of the solar system. This makes Bennu a time capsule, offering a window into the early solar system and potentially the prebiotic chemistry that preceded life on Earth.

Dante Lauretta, the principal investigator of OSIRIS-REx, underscores the significance of studying these materials, vital for comprehending the solar system's formation and the origins of life.

In the coming years, various laboratories will examine Bennu's sample, anticipating further insights into the origins and evolution of Earth-like planets.

Tools​

"Simplified Mobile App Creation for Shopify Stores"​

'Simplified Mobile App Creation for Shopify Stores'

LEO Mobile App Builder streamlines Shopify app development by offering a drag-and-drop interface, unlimited alerts, in-app filters, multi-currency support, and native language options. This tool empowers anyone to create a mobile app for their Shopify store without the need for coding.

Drag-and-drop interface: A user-friendly design method where elements are moved and placed on a screen using simple mouse actions, eliminating the need for complex coding.

Nuclei Templates for Automated Security Scanning​

"projectdiscovery/nuclei-templates" on GitHub is a collection of security testing templates. Organized by type—cloud, dns, http—these templates automate vulnerability scans. Essential files guide usage and contributions. With 43,949 commits since April 2020, it's a robust tool for security professionals.

GitHub is a platform for hosting and collaborating on code projects. A repository, or repo, is a storage space where a project's files and revision history are kept. Commits are updates or changes made to the files in a repository.

OpenAI's Whisper: A Multifunctional Speech Recognition Model​

OpenAI's Whisper: A Multifunctional Speech Recognition Model

Whisper, OpenAI's model, excels in speech tasks. It recognizes, translates, and identifies languages. Trained on diverse audio.

Multitasking, robust. Useful in speech technology. Resources available online.

  • Multilingual: Capable of understanding and processing multiple languages.
  • Model: A mathematical construct used to predict or understand data.
  • arXiv: An online archive for scientific papers before they are published.
  • Colab: Google's free cloud service for running Python code and Jupyter notebooks.

NATS Messaging System: Source Code and Documentation​

NATS, a messaging system, resides in the "nats-io/nats-server" GitHub repository. Managed by "nats-io," it is part of the Cloud Native Computing Foundation. It supports over 40 languages.

The repository includes:

  • Configuration, documentation, Docker files, and server components.
  • Markdown files on conduct, contributing, dependencies, and governance.
  • Active since 2012, with over 9,389 commits.

Key terms:

  • Cloud Native Computing Foundation (CNCF): A group promoting cloud-native applications.
  • Continuous Integration/Deployment (CI/CD): Practices automating software development processes.

Airclap: Fast, Offline File Transfer Across Devices​

Airclap: Fast, Offline File Transfer Across Devices

Airclap enables rapid file transfers without the need for internet. It operates seamlessly across all devices—Windows, Mac, and phones. Featuring a simple, modern aesthetic, it offers ultra-fast data transfers and maintains a record of sent files.

"Local Area Network" (LAN) links nearby devices without relying on the web. "Cross platform" denotes compatibility with any device. "Ultra fast" signifies file sending speeds that are notably quicker than average. "Modern User Interface" (UI) indicates a visually appealing and user-friendly design.

AI-Powered E-commerce Brand Management​

AI-Powered E-commerce Brand Management

Krut AI streamlines e-commerce by generating product images, crafting ad texts, editing visuals and copy, and speaking multiple languages. This tool simplifies online brand management, catering to a global audience.

Resource​

Twitter: Will​

Web Development Resources: https://github.com/riti2409/Resources_for_Web_Development

Twitter: Marc Lou​

Re Everything I learned as a solopreneur: How to find startup ideas, launch fast, and get profitable ↓ https://marclou.beehiiv.com/

Twitter: 小互​

"Teach you how to use AI to convert YouTube videos into SEO-friendly blog posts in just a few seconds https://xiaohu.ai/p/11161"

Twitter: Justin Welsh​

"Need some confidence to build your own lean, profitable, one-person business? Try my newsletter, The Saturday Solopreneur. Over 220,000 solopreneurs read it weekly. Join for free here: https://www.justinwelsh.me/subscribe"

Twitter: Pat Walls​

"Re nicely done @hieudinh" 👏 50 more examples of solo developers making at least $10K/month: https://www.starterstory.com/solo?utm_source=twitter&utm_campaign=compressx

Twitter: Wilson Wilson​

I can't believe this is available for free. @posthog has created an incredibly detailed handbook that reveals precisely how their company functions at every level. It covers engineering, design, marketing, operations, and more. There are tons of ideas to借鉴 for SaaS teams. Check it out at https://posthog.com/handbook.

Twitter: Justin Welsh​

Your website traffic is meaningless if nobody buys. Recently, I helped a friend boost conversions by 27% with a simple change. Tomorrow, I'm sharing my 5-4-3-2-1 method for creating irresistible offers. Join 220K to get the issue tomorrow: https://www.justinwelsh.me/subscribe

Twitter: Go​

With profile-guided optimization (PGO), you can assist the Go compiler in making more effective optimization decisions by providing it with a profile of your application at runtime. Discover how PGO operates and how to implement it, including on Google Cloud: https://cloud.google.com/blog/products/application-development/using-profile-guided-optimization-for-your-go-apps