Images credit goes to u/DigitalNightmare13 from Reddit

Himeka: original post

Ahko: original post

  • 45 Posts
  • 145 Comments
Joined 1 year ago
cake
Cake day: July 10th, 2024

help-circle



  • Someone with the expertise should correct me if I am wrong; it’s been 4-5 years since I learnt about NPUs during my internship so I am very rusty:

    You don’t even need a GPU if all you want to do is to run - i.e. perform inference with - a neural network (abbreviating it to NN). Just a CPU would do if the NN is sufficiently lightweight. The GPU is only needed to speed up the training of NNs.

    The thing is, the CPU is a general-purpose processor, so it won’t be able run the NN optimally / as efficiently as possible. Imagine you want to do something that requires the NN and as a result, you can’t do anything else on your phone / laptop (it won’t be problem for desktops with GPUs though).

    Where NPU really shines is when there are performance constraints on the model: when it has to be fast (to be specific: have real-time speed), lightweight and memory efficient. Use cases include mobile computing and IoT.

    In fact, there’s news about live translation on Apple AirPod. I think this may be the perfect scenario for using NPUs - ideally housed within the earphones directly but if not, within a phone.

    Disclaimer: I am only familiar with NPUs in the context of “old-school” convolutional neural networks (boy, tech moves so quickly). I am not familiar with NPUs for transformers - and LLMs by extension - but I won’t be surprised if NPUs have been adapted to work with them.











  • Those that I find the most useful are those that I (and likely many others) tend to take for granted.

    For example, fuzzy logic may very well be used in electronics that involve temperature control - fridge, aircon, rice cooker, water heater - under the hood.

    Another one is CSP (constraint-satisfaction problems) solvers which tend to be used in scheduling softwares. A possible use case is public transportation.

    There are probably lots more AIs working behind the scenes that benefit everyone, but don’t get the coverage because they are just boring tech now. People may not even consider them AI!

    I appreciate these AI for making my life so convenient.